Server Help Forum Index Server Help
Community forums for Subgame, ASSS, and bots
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   StatisticsStatistics   RegisterRegister 
 ProfileProfile   Login to check your private messagesLogin to check your private messages   LoginLogin (SSL) 

Server Help | ASSS Wiki (0) | Shanky.com
JPG/PNG Compression

 
Post new topic   Reply to topic Printable version
 View previous topic  Using CLT Post :: Post A good way to reading bits? (reading t...  View next topic  
Author Message
k0zy
Server Help Squatter


Gender:Gender:Male
Joined: Jan 11 2003
Posts: 571
Location: Germany
Offline

PostPosted: Fri Sep 01, 2006 3:44 am    Post subject: JPG/PNG Compression Reply to topic Reply with quote

As you might know I'm coding a LvzViewer.

So today I started to decompress files.
As the lvzFormat.txt states how to handle the *data* part:
Quote:
Read this all in, and then run it through zlib's decompress function.

That's what I did, when ZipLibSharp gave me an error message about missing zlib header.
JPGs and PNGs are not compressed at all in the lvz.

Why?
Even if it's just a few byte, it would result in smaller lvz files.

Did you forget to put that information in lvzFormat.txt, MGB?
Did PriitK forget to compress them?
Maybe it's forbidden to compresses them?
Or did just nobody notice?

DeBuildLevel handles the files just fine... Did zlib not complain about the header?
_________________
It's a shark! Oh my god! Unbelievable!
Back to top
View users profile Send private message Add User to Ignore List
Mine GO BOOM
Hunch Hunch
What What
Hunch Hunch<br>What What


Age:40
Gender:Gender:Male
Joined: Aug 01 2002
Posts: 3614
Location: Las Vegas
Offline

PostPosted: Fri Sep 01, 2006 3:57 am    Post subject: Re: JPG/PNG Compression Reply to topic Reply with quote

Bob Dole.. Bob Dole... Bob Dole...... bob dole.... bob... dole.... wrote:
That's what I did, when ZipLibSharp gave me an error message about missing zlib header.
JPGs and PNGs are not compressed at all in the lvz.

You need to use whatever is direct access to zlib's compress/decompress functions, not zip wrappers. There is no header info for it. It is a raw compressed stream, the same as .gz files. I've never used C# before, so I can't say. Google says that there are tons of wrappers out there, but for SharpZipLib, it looks like you should be using GZipInputStream for decompression. There is sample code they supply that looks like what you need (gzip, not zip).

As for the compression not being there, if you look at the FileTime field, if it is zero, there is no compression. Its weird, but thats how Continuum handles it, so thats how I made DeBuildLevel. You can have anything you want compressed, but the reason it isn't compress is that either the output stream is greater than the input (ie: you can't compress compressed data forever) or the time it would take to decompress the file isn't worth the time so it isn't compressed (ie: jpg/png usually). This is all upto the program that creates the LVZ file, because if you want to put a compressed file that is 1000 times larger the the original file, thats supported.
Back to top
View users profile Send private message Add User to Ignore List Send email
k0zy
Server Help Squatter


Gender:Gender:Male
Joined: Jan 11 2003
Posts: 571
Location: Germany
Offline

PostPosted: Fri Sep 01, 2006 4:32 am    Post subject: Re: JPG/PNG Compression Reply to topic Reply with quote

Mine GO BOOM wrote:
[..]


You need to use whatever is direct access to zlib's compress/decompress functions, not zip wrappers. There is no header info for it. It is a raw compressed stream, the same as .gz files. I've never used C# before, so I can't say. Google says that there are tons of wrappers out there, but for SharpZipLib, it looks like you should be using GZipInputStream for decompression. There is sample code they supply that looks like what you need (gzip, not zip).

As for the compression not being there, if you look at the FileTime field, if it is zero, there is no compression. Its weird, but thats how Continuum handles it, so thats how I made DeBuildLevel. You can have anything you want compressed, but the reason it isn't compress is that either the output stream is greater than the input (ie: you can't compress compressed data forever) or the time it would take to decompress the file isn't worth the time so it isn't compressed (ie: jpg/png usually). This is all upto the program that creates the LVZ file, because if you want to put a compressed file that is 1000 times larger the the original file, thats supported.


I don't understand.
I just looked at the data I read from the lvz again.
The only time the filetime is zero is when I read the objects.
I'm only able to tell, that the file has not been compressed if both sizes, before and after compression, are the same.

I know you can't compress compressed data forever.
However, the most PNGs and JPGs I found within lvz files compress just fine (I tried, they did get smaller).
And even if they get larger, it's not like 1000 times tongue.gif
I understand that in this case you rather store the original file.

I don't think gzip is able to decompress zip, because it's a whole diffrent algorithm. (Read gzip homepage)
And ZiplibSharp decompresses the files just fine with it's zlib port (not gzip)!
It doesn't give the "missing header" error on compressed files (all but jpg/png)! So this header is maybe where the dictionary is stored, that'd mean that even a raw stream would have that header.

I can decompress bmps, hlps, bm2s, but not png/jpg, because they simply aren't compressed, this has nothing to do with ZiplibSharp.

If you search google for C# and zlib the only things you find are:
zlib wrappers, and forum posts where users try to get zlib1.dll working but then just give up and rather use a wrapper.

EDIT: Reading your post again I think you misunderstood something.
ZipLibSharp is able to open zip files, but that's not what I used it for and I didn't use it's zip file methods.
Back to top
View users profile Send private message Add User to Ignore List
Bak
?ls -s
0 in


Age:24
Gender:Gender:Male
Joined: Jun 11 2004
Posts: 1826
Location: USA
Offline

PostPosted: Fri Sep 01, 2006 10:44 am    Post subject: Reply to topic Reply with quote

yeah, when I was making X-LvzToolkit I relized that lots of times the compressed size > uncompressed size, so in that case I leave it uncompressed. Another giveaway that it's uncompressed is if compressed size == uncompressed size (i think continuum looks for this, too).

In either case, if you think it's uncompressed, just output it to a file and see if you can open it in a picture editor.
_________________
SubSpace Discretion: A Third Generation SubSpace Client
Back to top
View users profile Send private message Add User to Ignore List AIM Address
k0zy
Server Help Squatter


Gender:Gender:Male
Joined: Jan 11 2003
Posts: 571
Location: Germany
Offline

PostPosted: Fri Sep 01, 2006 1:03 pm    Post subject: Reply to topic Reply with quote

Bak wrote:
yeah, when I was making X-LvzToolkit I relized that lots of times the compressed size > uncompressed size, so in that case I leave it uncompressed. Another giveaway that it's uncompressed is if compressed size == uncompressed size (i think continuum looks for this, too).

In either case, if you think it's uncompressed, just output it to a file and see if you can open it in a picture editor.


Which makes perfect sense.
It just seems to me that in some cases the png and jpg could be even smaller...
And this should be in lvzFormat.txt

Bob Dole.. Bob Dole... Bob Dole...... bob dole.... bob... dole....
Back to top
View users profile Send private message Add User to Ignore List
Smong
Server Help Squatter


Joined: 1043048991
Posts: 0x91E
Offline

PostPosted: Sat Sep 02, 2006 2:15 pm    Post subject: Reply to topic Reply with quote

I just uploaded all the dev files for my unreleased and unfinished lvz program:
http://forums.minegoboom.com/viewtopic.php?t=6686
Maybe it can help you.
Back to top
View users profile Send private message Add User to Ignore List Visit posters website MSN Messenger
grazzhoppa
Novice


Joined: Jan 03 2007
Posts: 29
Offline

PostPosted: Wed Jan 03, 2007 8:42 pm    Post subject: Reply to topic Reply with quote

How should these be handled:

File time is zero, name length is more than 0
---- bad section header? Should file times be guarenteed to be not zero when there is a file name?
File time is not zero, name length is 0
---- bad section header? The file does not have any name.

CLT and smong's unfinished lvz reader only check for, as I could see:
if(File Time being 0 and File Name length being 0)
to distinguish between the section being a file or map/screen object info.
Back to top
View users profile Send private message Add User to Ignore List
Bak
?ls -s
0 in


Age:24
Gender:Gender:Male
Joined: Jun 11 2004
Posts: 1826
Location: USA
Offline

PostPosted: Wed Jan 03, 2007 9:30 pm    Post subject: Reply to topic Reply with quote

Have you seen lvzFormat.txt?

Code: Show/Hide
After this, if the count is > 0, there should be a compressed header:
4-len str   Type - Should be 'CONT' also. If not, the compression header is bad, and do not use.
i32      Decompress Size - The size of the data after it would be decompressed. This is the value you should set your out-buffer for the zlib's compression call.
i32      File Time - This can either be a file time, if this compressed section is a file, or it could be 0. If 0, then is a special compressed section. The file time is the standard time_t format used by windows/unix.
i32      Compressed Size - The size of the compressed data in the .lvz. This is how many bytes you'll need to read to get your required data.
Null-end str   File Name - This string is always null ended. If no file time, the File Name should be blank (just the 0x00), but do not always count on it.
* data *   Compressed data - This should be the length of the Compressed size. Read this all in, and then run it through zlib's decompress function.
...
The decompressed data can be 1 of two types for the current .lvz format. The first type is more often found in .lvzs, and this is the file type. Files are determined by having a File Time not equal to 0, and having a File Name length not being 0. Please be warned, future formats of .lvz's may change one or the other of these checks, so a good safety feature would be that files require both of these to be true. Files are normally just saved to the proper folder and the correct file time set to it. If you do not know how to set file times, its is not 100% important for most DeBuildLevel like programs.

The only other current decompressed data type is the Object section. The object section is determined by having a File Time equal to 0 and the File Name length being 0.


I think the spec is silent about those two cases you mentioned.
Back to top
View users profile Send private message Add User to Ignore List AIM Address
grazzhoppa
Novice


Joined: Jan 03 2007
Posts: 29
Offline

PostPosted: Wed Jan 03, 2007 9:52 pm    Post subject: Reply to topic Reply with quote

That's why I'm asking for advice on it - its not defined icon_smile.gif

Should I try to extract the other sections in the .lvz when I encounter one of those situations or assume the whole file is bad?

Would a programmer who has made an .lvz creator ever allow those conditions to be written into an .lvz?
There might be a bug in some .lvz creation program that puts empty file names into the lvz. If this is true, then every section in the lvz except the one with an empty file name could still be good.

I've never written things that decode someone else's specification, and I'm asking for advice about handling those situations.
Back to top
View users profile Send private message Add User to Ignore List
Bak
?ls -s
0 in


Age:24
Gender:Gender:Male
Joined: Jun 11 2004
Posts: 1826
Location: USA
Offline

PostPosted: Thu Jan 04, 2007 12:43 am    Post subject: Reply to topic Reply with quote

print an error and quit or keep parsing... it's your choice, chances are you'll never come across an LVZ file like that anyway.
Back to top
View users profile Send private message Add User to Ignore List AIM Address
Animate Dreams
Gotta buy them all!
(Consumer whore)


Age:36
Gender:Gender:Male
Joined: May 01 2004
Posts: 821
Location: Middle Tennessee
Offline

PostPosted: Thu Jan 04, 2007 2:29 am    Post subject: Reply to topic Reply with quote

I was wondering, what language would this be coded in? And if it's not a cross-platform langugae, are you going to release a linux version? I was just thinking yesterday how much I would love one of these on Linux. >_>
Back to top
View users profile Send private message Add User to Ignore List Send email Visit posters website AIM Address MSN Messenger
Cyan~Fire
I'll count you!
I'll count you!


Age:36
Gender:Gender:Male
Joined: Jul 14 2003
Posts: 4608
Location: A Dream
Offline

PostPosted: Thu Jan 04, 2007 11:54 am    Post subject: Reply to topic Reply with quote

As it says in the linked post, C#. Which means a no-go for a Linux version.

(Well, I guess you could try Mono?)
_________________
This help is informational only. No representation is made or warranty given as to its content. User assumes all risk of use. Cyan~Fire assumes no responsibility for any loss or delay resulting from such use.
Wise men STILL seek Him.
Back to top
View users profile Send private message Add User to Ignore List Visit posters website
Bak
?ls -s
0 in


Age:24
Gender:Gender:Male
Joined: Jun 11 2004
Posts: 1826
Location: USA
Offline

PostPosted: Thu Jan 04, 2007 4:44 pm    Post subject: Reply to topic Reply with quote

what's wrong with X-LvzToolkit Animate?
Back to top
View users profile Send private message Add User to Ignore List AIM Address
D1st0rt
Miss Directed Wannabe


Age:36
Gender:Gender:Male
Joined: Aug 31 2003
Posts: 2247
Location: Blacksburg, VA
Offline

PostPosted: Thu Jan 04, 2007 8:15 pm    Post subject: Reply to topic Reply with quote

A lot of stuff will run under mono just with a simple recompile (provided you aren't referencing any binaries that you don't have the source to), though some minor modifications might need to be made
_________________

Back to top
View users profile Send private message Add User to Ignore List Visit posters website
Animate Dreams
Gotta buy them all!
(Consumer whore)


Age:36
Gender:Gender:Male
Joined: May 01 2004
Posts: 821
Location: Middle Tennessee
Offline

PostPosted: Fri Jan 05, 2007 4:59 am    Post subject: Reply to topic Reply with quote

Hmm, Bak, the biggest problem is having not heard about it before. Granted, I didn't do any searching - I just assumed that since he was making one, that one didn't already exist. I'll try this out in a bit.
Back to top
View users profile Send private message Add User to Ignore List Send email Visit posters website AIM Address MSN Messenger
Display posts from previous:   
Post new topic   Reply to topic    Server Help Forum Index -> LVZ/LVL Questions All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum
View online users | View Statistics | View Ignored List


Software by php BB © php BB Group
Server Load: 635 page(s) served in previous 5 minutes.

phpBB Created this page in 0.455805 seconds : 40 queries executed (90.9%): GZIP compression disabled