Server Help

BDwinsAlt · Posted: Fri Jan 05, 2007 8:44 pm Post subject: mapnewsdl trigger crashing

Ok, EVERY time the zone crashes it always proceeds a download. This is the last thing on log right before the zone recycled itself.

Jan 05 17:19:43 D <mapnewsdl> finished map/news download (transfer 00FB27E0)

How can I stop mapnewsdl from triggering this, can someone send me a modified version or something? That message comes up and right after it shows the log file opening and the modules loading, so right after map downloads its doing that, how can I prevent this behavior?

EDIT: This is an update of what happend just before the crash this time...

Code: Show/Hide

Jan 05 20:08:35 D <mapnewsdl> {dodgeball} [Enclose] sending map/lvz 6 (90065 bytes) (transfer 0208E7D8)
Jan 05 20:08:36 D <mapnewsdl> {dodgeball} [Knuzcano] sending map/lvz 0 (13597 bytes) (transfer 020DA198)
Jan 05 20:08:37 D <mapnewsdl> finished map/news download (transfer 020DA198)
Jan 05 20:08:37 D <mapnewsdl> finished map/news download (transfer 0208E7D8)
Jan 05 20:08:37 D <mapnewsdl> {dodgeball} [Knuzcano] sending map/lvz 6 (90065 bytes) (transfer 0208E7D8)
Jan 05 20:08:38 D <mapnewsdl> {dodgeball} [Splinta] sending map/lvz 0 (13597 bytes) (transfer 020DA198)
Jan 05 20:08:38 D <mapnewsdl> finished map/news download (transfer 020DA198)
Jan 05 20:08:38 D <mapnewsdl> {dodgeball} [Splinta] sending map/lvz 6 (90065 bytes) (transfer 020DA198)
Jan 05 20:08:41 D <mapnewsdl> finished map/news download (transfer 020DA198)
Jan 05 20:08:43 D <mapnewsdl> finished map/news download (transfer 0208E7D8)
Jan 05 20:08:45 D <mapnewsdl> {dodgeball} [Enclose] sending map/lvz 6 (90065 bytes) (transfer 0208E7D8)
Jan 05 20:08:55 D <mapnewsdl> finished map/news download (transfer 0208E7D8)
Jan 05 20:10:55 D <mapnewsdl> {dodgeball} [veolier] sending map/lvz 0 (13597 bytes) (transfer 02072E40)
Jan 05 20:10:55 D <mapnewsdl> finished map/news download (transfer 02072E40)
Jan 05 20:10:55 D <mapnewsdl> {dodgeball} [veolier] sending map/lvz 6 (90065 bytes) (transfer 02072E40)
Jan 05 20:10:57 D <mapnewsdl> finished map/news download (transfer 02072E40)
Jan 05 20:12:19 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/duckies/duckieships.lvz
Jan 05 20:12:19 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/duckies/_gfx.lvz
Jan 05 20:12:19 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/duckies/_sfx.lvz
Jan 05 20:12:23 D <mapnewsdl> {duckies} [Ooh lala.] sending map/lvz 2 (982674 bytes) (transfer 02076D48)
Jan 05 20:12:25 D <mapnewsdl> finished map/news download (transfer 02076D48)
Jan 05 20:13:14 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/duel/_gfx.lvz
Jan 05 20:13:14 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/duel/_sfx.lvz
Jan 05 20:13:14 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/duel/_ships.lvz
Jan 05 20:13:38 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/2v/_ships.lvz
Jan 05 20:13:38 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/2v/_gfx.lvz
Jan 05 20:13:38 W <mapnewsdl> compressed map/lvz is bigger than 256k: arenas/2v/_sfx.lvz
Jan 05 20:13:58 D <mapnewsdl> {2v2} [Splinta] sending map/lvz 0 (55393 bytes) (transfer 00FB9180)
Jan 05 20:14:02 D <mapnewsdl> finished map/news download (transfer 00FB9180)
Jan 05 20:15:23 D <mapnewsdl> {2v2} [stephan owns] sending map/lvz 0 (55393 bytes) (transfer 00FC6758)
Jan 05 20:15:29 D <mapnewsdl> finished map/news download (transfer 00FC6758)
Jan 05 20:22:16 D <mapnewsdl> {0} [Bleh! You're Dead!] sending map/lvz 0 (176477 bytes) (transfer 01089E78)
Jan 05 20:22:22 E <pymod> there are 2 remaining references to a player object!
Jan 05 20:22:35 D <mapnewsdl> finished map/news download (transfer 01089E78)
Jan 05 20:22:36 D <mapnewsdl> {0} [Bleh! You're Dead!] sending map/lvz 1 (660523 bytes) (transfer 00FB8360)

Mine GO BOOM · Posted: Fri Jan 05, 2007 11:35 pm Post subject:

Try running ASSS under a debugger? Visual Studio's debugger in Windows is a god sent, and gdb for Linux isn't that bad once you get used to it. Get a stack dump at the time of the crash to see what function(s) were in action when it crashed. Do you know which thread took the dump (networking, main, etc)?

BDwinsAlt · Posted: Fri Jan 05, 2007 11:39 pm Post subject:

I really don't know, we tried using bak's edited net module, makes downloads faster but the zone crashes all the same, I uploaded a map and lvz and right as it was done uploading I entered the arena and it crashed, I should have waited on that one I guess. I'm not sure if the server has a debugger, would a debugger keep the server running while logging the activity?

By the way the only custom C modules we have is bak's initial and net modules, which the zone was crashing before we added those and for the same reasons. Is there a common cause and fix for this?

EDIT:
Ok running in the debugger I immetiately see Heap Corruptions. I have it set to handle errors, maybe that means it will continue running even if an error is encountered, not sure.

Heap Corruption detected at: 01064218 01084188 0107F188... those are the first 3, there are about 20 in the first 5 minutes. I attached the debugger to ASSS.exe, so we'll see what happens.

In the post below log.txt is the output of a microsoft debugger as we made the zone crash by entering an arena and downloading the map, 2 people simply went into an arena once and downloaded and it did this.

BD · Guest Offline

Here is the log right when it crashed.

Cyan~Fire · Posted: Sat Jan 06, 2007 11:29 am Post subject:

That log is near useless.

So a heap corruption probably means a buffer overflow somewhere. Find out what variables are overflowing (by checking which ones are used in the crashing line) then look for any possible overflow with those variables. Oh, and once you find them tell us so we can look too.
_________________
This help is informational only. No representation is made or warranty given as to its content. User assumes all risk of use. Cyan~Fire assumes no responsibility for any loss or delay resulting from such use.
Wise men STILL seek Him.

BDwinsAlt · Posted: Sat Jan 06, 2007 1:46 pm Post subject:

Well the zone can be up for 17 hours and not crash until someone finally downloads a map and causes it to crash. Sometimes it crashes in a few minutes, a few hours, or almost wait an entire day. It's all linked to a mapnewsdl some how. I don't know which variable it is that is overflowing.

I'll run a test locally to see if its one of my modules or the ASSS core.

EDIT: I didn't get one heap error or anything in the debug log when I test it locally. So what's up? Could it be the fact that the server is running on 2 processors or something? My laptop is a 64 bit AMD with a Gig of ram, runs ASSS fine.

Maybe the linux version won't crash so easily, I'm not sure until I can test it for sure. Everything seems to work 100% without error on Windows XP Home/Pro. Weird.

Cyan~Fire · Posted: Sat Jan 06, 2007 7:05 pm Post subject:

Wait, is your laptop Linux? You'll definitely want to keep as much of the same setup between the two as possible.

If you have a debug build of ASSS running (you'll have to compile it in visual studio to make this work, which I've never tried), when it crashes you should be able to click the debug button (or "cancel" i think it is) and then see the exact line of code it's crashing on and do a whole bunch of debugging. Enjoy!

BDwinsAlt · Posted: Sat Jan 06, 2007 9:04 pm Post subject:

I have Windows XP Home on my laptop. I have ASSS running in vmware on my desktop as well. It works fine on both. Animate Dreams told me he had this issue before. He moved to linux and it stopped. Hopefully in feb we will be back on track. I'll try to get that asss debug thing.

Cyan~Fire · Posted: Sun Jan 07, 2007 12:53 pm Post subject:

Did you understand half of what I said? It's not an "asss debug thing," you need to compile ASSS with debug information. There are actually two options to do this; one I didn't mention in my previous post. Either you get ASSS compiling in Visual Studio (hard) and then debug in Visual Studio (easy), or compile ASSS in Dev-C++ (easy, specify "dbg=yes" or whatever the makefile option is) and then debug using Dev-C++ (harder than Visual Studio).

Solo@School · Guest Offline

Debug the asss thing!

It's not that hard to get ASSS running for VS, just step through it using the debugger to see what's going on. Well, fine, I guess it might be a pain if you have no clue.

BDwinsAlt · Posted: Mon Jan 08, 2007 8:04 am Post subject:

I am debugging it, zim was nice enough to make me a debug version and tell me how to log console output. I'll let you all know what happens as soon as the zone crashes.

Smong · Posted: Tue Jan 09, 2007 1:18 pm Post subject:

You could try setting "the affintity to one processor", something I recall reading somewhere. If that works it's only masking the problem, so keep looking for the source of it.
_________________
ss news

BDwinsAlt · Posted: Tue Jan 09, 2007 8:26 pm Post subject:

http://ss-halo.net/kent/debug.txt

This is the console output of the debug version of ASSS zim made it.

EDIT: From the logs zim said, "The problem seems to be in get_map(Arena *arena, int lvznum, int wantopt)."

So there was an issue in mapnewsdl. He sent me a debugger that will spit out more info on that part of mapnewsdl. When I get the fix I'll fill you in.