Results 1 to 5 of 5

Thread: fpaqvm (vm/jit)

  1. #1
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    377
    Thanks
    139
    Thanked 198 Times in 108 Posts

    fpaqvm (vm/jit)

    This compressor is based on fpaq0. It uses virtual machine to execute predictor. There are only linux executable files (x86), no soruce yet. fpaqvmnojit and fpaqvmjit. First one will execute user provided config file in virtual machine. File is translated into bytecode and then executed in vm. Second executable file translates input config file into bytecode and then emits x86 opcode and executes it.


    There are two config files. test1.cfg uses fpaq0 model. test2.cfg uses fpaqc model. test1.cfg is identical to original compressor. Second one is not. There are some differences in encoder etc.

    Config file language is subset of c.
    Code:
    $ ./fpaq0 c calgary.tar calgary.tar.fpaq0
    calgary.tar (3152896 bytes) -> calgary.tar.fpaq0 (1903024 bytes) in 1.02 s.
    $ ./fpaqvmjit c test1.cfg calgary.tar calgary.tar.fpaqvmjit
    calgary.tar (3152896 bytes) -> calgary.tar (1903024 bytes) in 2.74 s.
    $ ./fpaqvmjit c test2.cfg calgary.tar calgary.tar.fpaqvmjit
    calgary.tar (3152896 bytes) -> calgary.tar (1682796 bytes) in 1.79 s.
    $ ./fpaqvmnojit c test1.cfg calgary.tar calgary.tar.fpaqvmnojit
    calgary.tar (3152896 bytes) -> calgary.tar (1903024 bytes) in 53.29 s.
    $ ./fpaqvmnojit c test2.cfg calgary.tar calgary.tar.fpaqvmnojit
    calgary.tar (3152896 bytes) -> calgary.tar (1682796 bytes) in 35.59 s.
    Attached Files Attached Files
    KZo


  2. The Following 4 Users Say Thank You to kaitz For This Useful Post:

    Gonzalo (23rd December 2014),Matt Mahoney (23rd December 2014),Mike (22nd December 2014),nburns (22nd December 2014)

  3. #2
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Did you write your own C compiler? It seems like there is a lot of legal code (checked with gcc -c -Wall) that results in no errors but a size 0 output, like declaring a variable at the beginning of update() with an initial value like:

    int B=512; // fails
    int B; // OK

    It also fails if I try to change t from int* to an array, or to unsigned short*. Still, a very interesting idea to put a subset of C source code to decompress into a self describing archive. I was able to modify test2.cfg from order 0 to order 1 to improve compression, just as a quick test.

  4. #3
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    377
    Thanks
    139
    Thanked 198 Times in 108 Posts
    I did not write compiler myself. I'm using https://github.com/EarlGray/c4/ https://github.com/rswier/c4/ smalest i could find.
    Added for statement and removed some parts. As they are GPL i think i can use them.
    Arrays must be allocated before use with malloc, global variables can be used like:
    int a; //global
    int *b;
    void main(){
    int myzise;
    a=2; // in first usage
    myzise=a*10;
    b=malloc(myzise*sizeof(int));
    a=b[5];
    }
    int and char are only supported. Local variables must be at the start of function.

    I just wanted to see if its possible.
    KZo


  5. The Following User Says Thank You to kaitz For This Useful Post:

    Matt Mahoney (23rd December 2014)

  6. #4
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    377
    Thanks
    139
    Thanked 198 Times in 108 Posts
    Source code

    Code:
    $ ./fpaqvmjit c test1.cfg  enwik8 enwik8.0
    enwik8 (100000000 bytes) -> enwik8 (63391013 bytes) in 89.43 s.
    $ ./fpaqvmjit c test2.cfg  enwik8 enwik8.1
    enwik8 (100000000 bytes) -> enwik8 (61280454 bytes) in 59.27 s.
    $ ./fpaqvmjit c test3.cfg  enwik8 enwik8.2
    enwik8 (100000000 bytes) -> enwik8 (45979831 bytes) in 67.38 s.
    Attached Files Attached Files
    Last edited by kaitz; 29th December 2014 at 00:53.
    KZo


  7. The Following 2 Users Say Thank You to kaitz For This Useful Post:

    Bulat Ziganshin (29th December 2014),Matt Mahoney (29th December 2014)

  8. #5
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    377
    Thanks
    139
    Thanked 198 Times in 108 Posts
    test4.cfg o2
    Code:
    $ ./fpaqvmjit c test4.cfg  enwik8 enwik8.3
    enwik8 (100000000 bytes) -> enwik8 (36950806 bytes) in 75.33 s.
    test3.cfg is o1 not o2.
    Attached Files Attached Files
    KZo


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •