Page 1 of 2 12 LastLast
Results 1 to 30 of 31

Thread: Quasi-Unlimited Packer with no data loss ( seems serious )

  1. #1
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Cool Quasi-Unlimited Packer with no data loss ( seems serious )

    this is a yabasic programc;

    try it on small size zip files ( like as of 1Mb ) to check its efficiency

    then a serious programmer will enhance my program for professional use

    dim t(100000000)
    dim u(100000000)
    dim v(1500)

    i=1

    open 1,"d:dummy.zip","rb"
    open 2,"d:dummy1.txt","wb"
    open 3,"d:dummy1_log.txt","w"

    after each execution of the program, pack the file dummy1.txt which will label dummy1.zip, and then a dummy2.txt will appear

    actually, I used this method on a 800 kb zip file which, after 15 executions, drops to 4 kb...

    give it a try, I hope it will enlight your journey

    while(!eof(1))
    t(i)=peek(1)
    i=i+1
    rem print i
    wend

    close 1

    20
    print "pass 1..."

    j=1
    k=1
    o=0
    m=1

    if (t(1)-128>=0) then
    f=1
    else
    f=0
    endif

    while (j<=i)
    rem print j,t(j)
    zz=256
    while (zz>1)
    zz=zz/2
    if (t(j)-zz>=0) then
    40
    f=1
    else
    f=0
    endif
    if (f=1 and o=0) then
    k=k+1
    u(k)=1
    o=1
    else if (f=1 and o=1) then
    u(k)=u(k)+1
    if (m<u(k)) then
    m=u(k)
    endif
    t(j)=t(j)-zz
    elseif (f=0 and o=0) then
    u(k)=u(k)+1
    if (m<u(k)) then
    m=u(k)
    endif
    60
    elseif (f=0 and o=1) then
    k=k+1
    u(k)=1
    o=0
    endif
    endif
    wend
    j=j+1
    wend
    print "almost finished"

    print
    print m

    for l=1 to k
    poke #2,u(l)
    v(u(l))=v(u(l))+1

    next l
    80
    close 2

    print "finish"

    for m=1 to 1500
    if (v(m)>0) then
    print m,v(m)
    r=r+m*v(m)
    print #3,m,v(m),m*v(m)
    endif
    next m

    print #3,r
    close 3

    end



  2. #2
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    decompressor implementation is obvious, so it left as exercise to the reader

  3. The Following 2 Users Say Thank You to Bulat Ziganshin For This Useful Post:

    encode (15th November 2016),PSHUFB (18th January 2017)

  4. #3
    Member
    Join Date
    May 2009
    Location
    France
    Posts
    95
    Thanks
    13
    Thanked 72 Times in 42 Posts

  5. #4
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    hint: the last line from dummy_log is, with a one-byte difference, exactly 8 times the original dummy.zip; that is to say, multiply the file size by 8 and you will get the bit_original_filesize...

    ( obscure, isn't it ?? )

  6. #5
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by uglyvan View Post
    I used this method on a 800 kb zip file which, after 15 executions, drops to 4 kb...
    I tried it with a zipped yabasic.exe as input:

    yabasic.exe 372,224 bytes
    dummy.zip 186,836 bytes
    dummy1.txt 360,001 bytes
    dummy1.zip 134,730 bytes
    dummy2.txt 261,831 bytes
    dummy2.zip 96,542 bytes
    dummy3.txt 187,265 bytes
    dummy3.zip 69,726 bytes
    dummy4.txt 135,351 bytes
    dummy4.zip 50,280 bytes
    dummy5.txt 97,539 bytes
    dummy5.zip 36,543 bytes
    dummy6.txt 70,869 bytes
    dummy6.zip 26,695 bytes
    dummy7.txt 51,685 bytes
    dummy7.zip 19,672 bytes
    dummy8.txt 38,099 bytes
    dummy8.zip 14,665 bytes
    dummy9.txt 28,345 bytes
    dummy9.zip 10,928 bytes
    dummy10.txt 21,137 bytes
    dummy10.zip 8,329 bytes
    dummy11.txt 16,017 bytes
    dummy11.zip 6,421 bytes
    dummy12.txt 12,337 bytes
    dummy12.zip 5,012 bytes
    dummy13.txt 9,625 bytes
    dummy13.zip 3,967 bytes
    dummy14.txt 7,601 bytes
    dummy14.zip 3,180 bytes
    dummy15.txt 6,087 bytes
    dummy15.zip 2,575 bytes

    How to reverse it

  7. #6
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    to bring on the unpacker we need to know the program's philosophy;
    here it comes:

    1° it acts as a complement to any classic packer ( i.e. winzip etc )

    ir is based on words, ranging from 1bit to n bits;
    a word is anything like that:

    say you have 3 bytes of data,
    137-22-49
    137=128+0+0+0+8+0+0+1
    22=0+0+0+1+0+1+2+0
    49=0+0+1+1+0+0+0+1

    now merging data
    128+0+0+0+8+0+0+1+0+0+0+1+0+1+2+0+0+0+1+1+0+0+0+1

    first byte of the file=1 ( 128=8th bit=1 )
    1+3+1+2+1+3+1+1+2+3+2+3+1

    then print to disk
    1st byte = sign of the 1st bit
    2nd byte = 1
    3rd byte = 3
    then 1
    2
    1
    3
    1
    1
    2
    3
    2
    3
    1


    ( grmpf! hope it doesn't confuse you, you easy readers )

  8. #7
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by uglyvan View Post
    now merging data
    128+0+0+0+8+0+0+1+0+0+0+1+0+1+2+0+0+0+1+1+0+0+0+1

    first byte of the file=1 ( 128=8th bit=1 )
    1 + 3+1+2 +1 +3+1+1 +2+3 +2 +3 +1
    One 1 become 1 and one zero become 1
    How you can distinct with decoding 1 from one 1 from 1 from one zero?

    Two 1's become 2 and two zero's become 2
    How you can distinct with decoding 2 from two 1's from 2 from two zero's?

  9. #8
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    Sportman, runs of 0s and 1s are going after each other

    i guess that he is using non-prefix code to encode lengths

  10. #9
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    wtf??

    rename words to n-length bits

    with same value like in 255=1+1+1+1+1+1+1+1
    now if your file begins with a 255 1st word ( after assigning a 1 for the value of the 8th bit ( 255=128+64+32+16+8+4+2+1 ) will have a min value of 8;
    say 2nd byte is 137
    137=128 (8th bit set )+ 8 ( 4th bit set ) + 1 ( 1st bit set )
    137=1+0+0+0+1+0+0+1
    merge 255+137
    1111111110001001
    now writing to disk
    111111111=9
    000=3
    1=1
    00=2
    1=1

    grmpf ugly is bad teacher :/

  11. #10
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    ... by the way ...

    some of them say " it's worth a rough 25 Billion Dollars !! "

    - and how much money for little Yvan ??

    - hmpf! 4 Billions .

  12. #11
    Member
    Join Date
    Aug 2016
    Location
    USA
    Posts
    41
    Thanks
    9
    Thanked 16 Times in 11 Posts
    Finally, we've defeated the pigeonhole principle! Take that, Dirichlet!

  13. #12
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    very good! thank you!!

  14. #13
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    Sportman, runs of 0s and 1s are going after each other
    I knew that, but I talked about how to determine the first value type of the start encode byte, because in his examples he also encode the start zero value(s), you can skip them but then you need to know how many where skipped.

    Edit:
    Looks like he stream it, then only the first value type need to be known, I see he named it "sign of the 1st bit".
    Last edited by Sportman; 14th November 2016 at 21:54.

  15. #14
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by uglyvan View Post
    and how much money for little Yvan ??
    How old are you?

  16. #15
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by uglyvan View Post
    some of them say " it's worth a rough 25 Billion Dollars !! "
    It can make some people in this forum jobless.

  17. #16
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    437
    Thanks
    137
    Thanked 152 Times in 100 Posts
    Give up - infinite compression has already been done. Sorry, but you're slow to the market.

    https://github.com/philipl/pifs


  18. #17
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Dummy14.zip start with:
    0101 0000 0100 1011 0000 0011 0000 0100 0001 0100 0000 0000 0000 0000 0000 0000 0000

    I expect:
    (0) 1 1 1 1 5 1 2 1 1 2 6 2 5 1 5 1 1 1 30

    But dummy15.txt start with:
    1 7 1 7 6 2 5 3 3 5 28 4 10 6 1 7 1 7 1 7 2 6 4 4 3 5 1 9 1 5 4 4
    Last edited by Sportman; 15th November 2016 at 11:07. Reason: Source value fix

  19. #18
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Sportman View Post
    Dummy14.zip start with:
    0101 0000 0100 1011 0000 0011 0000 0100 0000 0111 0000 0000 0000 0000 0000 0000

    I expect:
    (0) 1 1 1 1 5 1 2 1 1 2 6 2 5 1 7 3 24

    But dummy15.txt start with:
    1 7 1 7 6 2 5 3 3 5 28 4 10 6 1 7 1 7 1 7 2 6 4 4 3 5 1 9 1 5 4 4
    well,if my program is broken, just write your own using the philosophy I posted above; a program written in 100 lines can't be hard to rebuild

  20. #19
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Sportman View Post
    How old are you?
    well, if you wanna see my face, check out "uglyvan iron maiden piece of mind" on youtube

  21. #20
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Sportman View Post
    Dummy14.zip start with:
    0101 0000 0100 1011 0000 0011 0000 0100 0000 0111 0000 0000 0000 0000 0000 0000

    I expect:
    (0) 1 1 1 1 5 1 2 1 1 2 6 2 5 1 7 3 24

    But dummy15.txt start with:
    1 7 1 7 6 2 5 3 3 5 28 4 10 6 1 7 1 7 1 7 2 6 4 4 3 5 1 9 1 5 4 4
    no no, there is no error here:

    each word is assigned to one byte

    1+7=8
    1+7=8
    6+2=8
    5+3=8
    3+5=8

  22. #21
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by uglyvan View Post
    no no, there is no error here
    Then I do not understand what you do.

    When I do what I thought you do, then it's almost the same I code 15 years ago, I found some old code (without the starting type) and got:

    dummy.zip 186,836 bytes to byte values 739,391 bytes with different statistics and different value list then your dummy1.txt 360,001 bytes.

    Compressing:
    236,690 bytes zip
    186,686 bytes paq8pxd18
    186,582 bytes emma
    186,162 bytes cmix

  23. #22
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Sportman View Post
    Then I do not understand what you do.

    When I do what I thought you do, then it's almost the same I code 15 years ago, I found some old code (without the starting type) and got:

    dummy.zip 186,836 bytes to byte values 739,391 bytes with different statistics and different value list then your dummy1.txt 360,001 bytes.

    Compressing:
    236,690 bytes zip
    186,686 bytes paq8pxd18
    186,582 bytes emma
    186,162 bytes cmix
    then, my program must be broken

  24. #23
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    856
    Thanks
    45
    Thanked 104 Times in 82 Posts
    Quote Originally Posted by uglyvan View Post
    then, my program must be broken
    Says kinda in the title "Unlimited Packer"...

  25. #24
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    ... or, if you prefer, I have absolutely no knowledge in the fields of the business ...

  26. #25
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by uglyvan View Post
    ... or, if you prefer, I have absolutely no knowledge in the fields of the business ...
    shall I drop it as a GPL product ?

  27. #26
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by uglyvan View Post
    I have absolutely no knowledge in the fields of the business
    None of the persons who got something similar working, so never stop trying and LawCounsels shall be happy with this outcome

  28. #27
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    Honestly I have not well understood if the program manages to actually compress already compressed data. If so it is right to give help to those who had this idea.

  29. #28
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by Nania Francesco View Post
    Honestly I have not well understood if the program manages to actually compress already compressed data.
    No it always expand data, but expanded weak compressed data can be one time compressed smaller with a better compressor, after that this process can not be repeated with the same outcome, so it do not work.

  30. #29
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by uglyvan View Post
    ... by the way ...

    some of them say " it's worth a rough 25 Billion Dollars !! "

    - and how much money for little Yvan ??

    - hmpf! 4 Billions .
    You're in the army now

  31. #30
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    hi folks !

    ( if you wanna make money with my stuff, I claim the 15% droits d'auteur; if worth 25 Billion then it rounds to approximately 4 Billion )


Page 1 of 2 12 LastLast

Similar Threads

  1. Machine Learning to identify weight loss parameters
    By Sportman in forum The Off-Topic Lounge
    Replies: 0
    Last Post: 13th August 2016, 13:10
  2. ( almost ) unlimited compression with no data loss
    By uglyvan in forum Data Compression
    Replies: 4
    Last Post: 15th April 2016, 15:32
  3. Looking for a low memory usage .exe packer
    By SvenBent in forum Data Compression
    Replies: 7
    Last Post: 5th April 2015, 17:19
  4. Does Any Body Have Dir2pak packer ?
    By Manjunath in forum Download Area
    Replies: 14
    Last Post: 20th December 2014, 18:12
  5. rash - dummy EXE packer
    By encode in forum Forum Archive
    Replies: 17
    Last Post: 26th January 2008, 12:27

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •