Server Help

LVZ/LVL Questions - JPG/PNG Compression
k0zy - Fri Sep 01, 2006 3:44 am
Post subject: JPG/PNG Compression
As you might know I'm coding a LvzViewer.

So today I started to decompress files.
As the lvzFormat.txt states how to handle the *data* part:
Quote:
Read this all in, and then run it through zlib's decompress function.

That's what I did, when ZipLibSharp gave me an error message about missing zlib header.
JPGs and PNGs are not compressed at all in the lvz.

Why?
Even if it's just a few byte, it would result in smaller lvz files.

Did you forget to put that information in lvzFormat.txt, MGB?
Did PriitK forget to compress them?
Maybe it's forbidden to compresses them?
Or did just nobody notice?

DeBuildLevel handles the files just fine... Did zlib not complain about the header?
Mine GO BOOM - Fri Sep 01, 2006 3:57 am
Post subject: Re: JPG/PNG Compression
Bob Dole.. Bob Dole... Bob Dole...... bob dole.... bob... dole.... wrote:
That's what I did, when ZipLibSharp gave me an error message about missing zlib header.
JPGs and PNGs are not compressed at all in the lvz.

You need to use whatever is direct access to zlib's compress/decompress functions, not zip wrappers. There is no header info for it. It is a raw compressed stream, the same as .gz files. I've never used C# before, so I can't say. Google says that there are tons of wrappers out there, but for SharpZipLib, it looks like you should be using GZipInputStream for decompression. There is sample code they supply that looks like what you need (gzip, not zip).

As for the compression not being there, if you look at the FileTime field, if it is zero, there is no compression. Its weird, but thats how Continuum handles it, so thats how I made DeBuildLevel. You can have anything you want compressed, but the reason it isn't compress is that either the output stream is greater than the input (ie: you can't compress compressed data forever) or the time it would take to decompress the file isn't worth the time so it isn't compressed (ie: jpg/png usually). This is all upto the program that creates the LVZ file, because if you want to put a compressed file that is 1000 times larger the the original file, thats supported.
k0zy - Fri Sep 01, 2006 4:32 am
Post subject: Re: JPG/PNG Compression
Mine GO BOOM wrote:
[..]


You need to use whatever is direct access to zlib's compress/decompress functions, not zip wrappers. There is no header info for it. It is a raw compressed stream, the same as .gz files. I've never used C# before, so I can't say. Google says that there are tons of wrappers out there, but for SharpZipLib, it looks like you should be using GZipInputStream for decompression. There is sample code they supply that looks like what you need (gzip, not zip).

As for the compression not being there, if you look at the FileTime field, if it is zero, there is no compression. Its weird, but thats how Continuum handles it, so thats how I made DeBuildLevel. You can have anything you want compressed, but the reason it isn't compress is that either the output stream is greater than the input (ie: you can't compress compressed data forever) or the time it would take to decompress the file isn't worth the time so it isn't compressed (ie: jpg/png usually). This is all upto the program that creates the LVZ file, because if you want to put a compressed file that is 1000 times larger the the original file, thats supported.


I don't understand.
I just looked at the data I read from the lvz again.
The only time the filetime is zero is when I read the objects.
I'm only able to tell, that the file has not been compressed if both sizes, before and after compression, are the same.

I know you can't compress compressed data forever.
However, the most PNGs and JPGs I found within lvz files compress just fine (I tried, they did get smaller).
And even if they get larger, it's not like 1000 times tongue.gif
I understand that in this case you rather store the original file.

I don't think gzip is able to decompress zip, because it's a whole diffrent algorithm. (Read gzip homepage)
And ZiplibSharp decompresses the files just fine with it's zlib port (not gzip)!
It doesn't give the "missing header" error on compressed files (all but jpg/png)! So this header is maybe where the dictionary is stored, that'd mean that even a raw stream would have that header.

I can decompress bmps, hlps, bm2s, but not png/jpg, because they simply aren't compressed, this has nothing to do with ZiplibSharp.

If you search google for C# and zlib the only things you find are:
zlib wrappers, and forum posts where users try to get zlib1.dll working but then just give up and rather use a wrapper.

EDIT: Reading your post again I think you misunderstood something.
ZipLibSharp is able to open zip files, but that's not what I used it for and I didn't use it's zip file methods.
Bak - Fri Sep 01, 2006 10:44 am
Post subject:
yeah, when I was making X-LvzToolkit I relized that lots of times the compressed size > uncompressed size, so in that case I leave it uncompressed. Another giveaway that it's uncompressed is if compressed size == uncompressed size (i think continuum looks for this, too).

In either case, if you think it's uncompressed, just output it to a file and see if you can open it in a picture editor.
k0zy - Fri Sep 01, 2006 1:03 pm
Post subject:
Bak wrote:
yeah, when I was making X-LvzToolkit I relized that lots of times the compressed size > uncompressed size, so in that case I leave it uncompressed. Another giveaway that it's uncompressed is if compressed size == uncompressed size (i think continuum looks for this, too).

In either case, if you think it's uncompressed, just output it to a file and see if you can open it in a picture editor.


Which makes perfect sense.
It just seems to me that in some cases the png and jpg could be even smaller...
And this should be in lvzFormat.txt

Bob Dole.. Bob Dole... Bob Dole...... bob dole.... bob... dole....
Smong - Sat Sep 02, 2006 2:15 pm
Post subject:
I just uploaded all the dev files for my unreleased and unfinished lvz program:
http://forums.minegoboom.com/viewtopic.php?t=6686
Maybe it can help you.
grazzhoppa - Wed Jan 03, 2007 8:42 pm
Post subject:
How should these be handled:

File time is zero, name length is more than 0
---- bad section header? Should file times be guarenteed to be not zero when there is a file name?
File time is not zero, name length is 0
---- bad section header? The file does not have any name.

CLT and smong's unfinished lvz reader only check for, as I could see:
if(File Time being 0 and File Name length being 0)
to distinguish between the section being a file or map/screen object info.
Bak - Wed Jan 03, 2007 9:30 pm
Post subject:
Have you seen lvzFormat.txt?

Code: Show/Hide
After this, if the count is > 0, there should be a compressed header:
4-len str   Type - Should be 'CONT' also. If not, the compression header is bad, and do not use.
i32      Decompress Size - The size of the data after it would be decompressed. This is the value you should set your out-buffer for the zlib's compression call.
i32      File Time - This can either be a file time, if this compressed section is a file, or it could be 0. If 0, then is a special compressed section. The file time is the standard time_t format used by windows/unix.
i32      Compressed Size - The size of the compressed data in the .lvz. This is how many bytes you'll need to read to get your required data.
Null-end str   File Name - This string is always null ended. If no file time, the File Name should be blank (just the 0x00), but do not always count on it.
* data *   Compressed data - This should be the length of the Compressed size. Read this all in, and then run it through zlib's decompress function.
...
The decompressed data can be 1 of two types for the current .lvz format. The first type is more often found in .lvzs, and this is the file type. Files are determined by having a File Time not equal to 0, and having a File Name length not being 0. Please be warned, future formats of .lvz's may change one or the other of these checks, so a good safety feature would be that files require both of these to be true. Files are normally just saved to the proper folder and the correct file time set to it. If you do not know how to set file times, its is not 100% important for most DeBuildLevel like programs.

The only other current decompressed data type is the Object section. The object section is determined by having a File Time equal to 0 and the File Name length being 0.


I think the spec is silent about those two cases you mentioned.
grazzhoppa - Wed Jan 03, 2007 9:52 pm
Post subject:
That's why I'm asking for advice on it - its not defined icon_smile.gif

Should I try to extract the other sections in the .lvz when I encounter one of those situations or assume the whole file is bad?

Would a programmer who has made an .lvz creator ever allow those conditions to be written into an .lvz?
There might be a bug in some .lvz creation program that puts empty file names into the lvz. If this is true, then every section in the lvz except the one with an empty file name could still be good.

I've never written things that decode someone else's specification, and I'm asking for advice about handling those situations.
Bak - Thu Jan 04, 2007 12:43 am
Post subject:
print an error and quit or keep parsing... it's your choice, chances are you'll never come across an LVZ file like that anyway.
Animate Dreams - Thu Jan 04, 2007 2:29 am
Post subject:
I was wondering, what language would this be coded in? And if it's not a cross-platform langugae, are you going to release a linux version? I was just thinking yesterday how much I would love one of these on Linux. >_>
Cyan~Fire - Thu Jan 04, 2007 11:54 am
Post subject:
As it says in the linked post, C#. Which means a no-go for a Linux version.

(Well, I guess you could try Mono?)
Bak - Thu Jan 04, 2007 4:44 pm
Post subject:
what's wrong with X-LvzToolkit Animate?
D1st0rt - Thu Jan 04, 2007 8:15 pm
Post subject:
A lot of stuff will run under mono just with a simple recompile (provided you aren't referencing any binaries that you don't have the source to), though some minor modifications might need to be made
Animate Dreams - Fri Jan 05, 2007 4:59 am
Post subject:
Hmm, Bak, the biggest problem is having not heard about it before. Granted, I didn't do any searching - I just assumed that since he was making one, that one didn't already exist. I'll try this out in a bit.
All times are -5 GMT
View topic
Powered by phpBB 2.0 .0.11 © 2001 phpBB Group