Results 1 to 5 of 5

Thread: How to build Bonfield's rANS coders on windows

  1. #1
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,134
    Thanks
    179
    Thanked 921 Times in 469 Posts

    How to build Bonfield's rANS coders on windows

    https://github.com/jkbonfield/rans_static
    http://nishi.dreamhosters.com/u/rans_bonfield_v0.rar

    Fixes:
    - added -D__USE_MINGW_ANSI_STDIO to compile printf %lld
    - "-include setbinary.inc" to support stdin/stdout
    - some printf(...) changed to fprintf(stderr,...)
    - increased stack size with -Wl,--stack,16777216, with default 2M stack -o1 works only in arith_static
    - replaced -mtune=native with -march=k8; "native" would be AVX512 here.

    Some exes (r8*) are designed for specific platforms, so might not work

    Usage:
    rANS_static BOOK1 BOOK1.rans
    rANS_static -o1 BOOK1 BOOK1.rans
    rANS_static -d BOOK1.rans BOOK1.unp
    rANS_static -o1 -8 BOOK1 BOOK1.rans -- change renorm_size to 8
    rANS_static -d -8 BOOK1.rans BOOK1.unp -- have to specify for decoder too, since its not written to file unlike order

  2. The Following 5 Users Say Thank You to Shelwien For This Useful Post:

    Bulat Ziganshin (3rd May 2019),encode (4th May 2019),JamesB (3rd May 2019),load (3rd May 2019),Mike (4th May 2019)

  3. #2
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    437
    Thanks
    137
    Thanked 152 Times in 100 Posts
    I'm rather embarrassed by the poor state of that code. It massively needs overhauling and moving bits to common functions. Definite cargo-cult there: cut, paste, hack

    Is there 8 there a build of -DNX=8 for 8-way unrolling, or some ultra large word size for the renorm step?

  4. #3
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,134
    Thanks
    179
    Thanked 921 Times in 469 Posts
    Yes, there're some exes with NX=8 or more.
    But all of them seem to support "-8" option and produce different results with it, so I added it to description.

    The code is actually more portable that most other libs.
    I usually have to edit sources to get stuff to compile on windows, while here I didn't even get much warnings.
    Half of it didn't work then, but I guess you never tested it on windows :)

  5. #4
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    437
    Thanks
    137
    Thanked 152 Times in 100 Posts
    Indeed not, but I have Windows at home so I ought to give it a try.

    Regardless I'd like to try a large-unrolled AVX2 version (r32x16b_avx2 is the most complete IIRC) in CRAM to see if it justifies the effort in getting it ship-shape and using it in production. If so, it'll be a requirement for it to be reasonably portable. At present though I think only around 1/3rd of my CPU is entropy encoding, so that set a limit of how much I can gain. Not worth it unless I'm breaking the spec anyway (which I'm working on atm).

  6. #5
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,134
    Thanks
    179
    Thanked 921 Times in 469 Posts
    1. With so many streams there's visible redundancy on flush.
    Why not compress flush states? And also frequency tables?

    2. What do you think about potential of non-standard rANS implementation?
        uint32_t x = *r;
    x = freq * (x >> scale_bits) + (x & mask) - start;
    Here the dependency on "x" in (x&mask) is not necessary - we can just as well use bits from another coder instance.

Similar Threads

  1. Benchmarking Entropy Coders
    By dnd in forum Data Compression
    Replies: 183
    Last Post: 27th June 2018, 13:48
  2. How to build a developer blog / download site the easy way?
    By packDEV in forum The Off-Topic Lounge
    Replies: 1
    Last Post: 10th January 2014, 22:17
  3. Lossless image coders
    By Madgeniy in forum Data Compression
    Replies: 26
    Last Post: 11th July 2011, 09:06
  4. Comparison of the recent CM demo coders
    By Shelwien in forum Data Compression
    Replies: 38
    Last Post: 13th June 2008, 13:21
  5. I cannot find jpegoptim win32 build
    By SvenBent in forum Forum Archive
    Replies: 2
    Last Post: 30th November 2007, 23:41

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •