Pocket Heaven

kuwanger · Member Joined: 11 Mar 2004 Posts: 278

Warning - Alpha Release

Yep, I've decided to do another PocketNES version with compressed file support. There are some rather major changes to the design, though. Here's a list:

1. Only supports up to 128KB ROMs (to avoid all sorts of nasty stuff)
2. Now works with GBA LZ77 compressed files too (use extension .l77)
3. Only works with pogoshell (this is unlikely to change given how much of a hassle it is to getting compilations to work well with compressed files)

Having stated all that, this is my first go at reproducing compression support with PocketNES, so I'd assume there's going to be at least a few bugs. You can use the old pocketnes-k's nez_cmp program to compress files to .nez and my own d or other programs like dcmp and GBACrusher for .l77. They're not included with the binary+source.

Note: This is primarily a test release. I've not done extensive testing to see if save states and the like are broken (though I doubt they are). Please reply with any positive/negative comments.

Binary and Source
_________________
Kuwanger Projects Like stag beetles in tubes

tepples · Posted: Sun Sep 11, 2005 6:49 am Post subject: Re: Unofficial PocketNES 9.98-k

Dwedit · Posted: Sun Sep 11, 2005 6:53 am Post subject:

Apack compresses a lot better, maybe you could merge in some parts of pocketnes_compy into this?
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!

kuwanger · Member Joined: 11 Mar 2004 Posts: 278

@tepples: Yea, any gba lz77 compression program will work. I was just listing a few.

@dwedit: Is there an open apack compressor out there? I'd rather not include decompression support when I can't legally include the compressor (and hopefully source). I need to port the lzo compressor, anyways, to java anyways to avoid the whole cross-platform compiling thing; if I include apack, I'd want to do the same.
_________________
Kuwanger Projects Like stag beetles in tubes

kuwanger · Member Joined: 11 Mar 2004 Posts: 278

I've done some more testing. Using lzo1x_1 (the standard/only compressor in minilzo), I get an average of ~68.5% the original size. Using lzo1x_999 (from lzo), I get an average of ~60.7% the original size. Just so you know, all the lzo1x compressors are compatible. So, it'd seem the best thing to do would be to distribute the lzo1x_999 compressor.
_________________
Kuwanger Projects Like stag beetles in tubes

Dwedit · Posted: Mon Sep 12, 2005 4:00 am Post subject:

What's the maximum lookback size on those LZ compressors?
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!

tepples · Posted: Mon Sep 12, 2005 4:54 am Post subject:

I can't say about the others, but GBA LZ77 uses a window of about 4 KB and runlengths of 18 bytes.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

Dwedit · Posted: Tue Sep 13, 2005 6:38 pm Post subject:

Is there anything stopping someone from taking an existing LZ compressor (which uses a large window size) and generating an aplib compatible output stream?
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!

tepples · Posted: Tue Sep 13, 2005 10:00 pm Post subject:

Dwedit · Posted: Tue Sep 13, 2005 10:48 pm Post subject:

The window size is infinate. The gamma code used allows it to keep shifting left and adding 0/1 until it hits a zero bit indicating not to continue.

Pretty much once it knows that it's reading a gamma code, it follows this simple algorithm:
start with 2
* read a bit
* add the bit
* read another bit
* if 1, multiply by 2 and repeat
* if 0, the number is done.
So big numbers are represented in 2(n-1) bits.
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!

kuwanger · Member Joined: 11 Mar 2004 Posts: 278

I've been working on creating an apack compressor, based on the apack.s decompressor included with pocketnes_compy. So far, I've written a C version of apack.s, an apack compressor that doesn't work (unapack segfaults on trying to decompress what it produces; also, the output file is not optimal), and a pair of compressors/decompressors that are modelled after apack but are simplified and I've verified that work. This last pair I call kpack/unkpack. They're more a stop-gap until a proper apack compressor is built, to better understand how apack works. So, I'd appreciate any help on getting my apack.c program to work well, let alone at all.

What I've got so far: Binary and Source

Hopefully we will get an apack compressor working soon, and then I can properly incorporate the compressor into my pocketnes-k package. This of course assumes I can get that working properly. Smile

_________________
Kuwanger Projects Like stag beetles in tubes

Dwedit · Posted: Wed Sep 21, 2005 1:27 am Post subject:

There's also the big guns: LZMA. It's what 7-Zip uses. Appearently, it only requires 16k/32k (getting contradictory numbers here) of additional memory to decompress a file.
I think an easy way to get 32k of memory would be to use GBA_VRAM, and blank the screen during decompression, then restore the original content afterwards.

It might be really slow though, haven't speed tested it yet.
Edit: The docs suggest it would take about 3 seconds to decompress 256k. Still haven't speed tested it yet.
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!