The byte order should be the same, as both systems are 32-bit Intel x86.
Also it just seems like the its doing the decryption fine, but then not passing the value back to ASSS. Instead it uses the value originally passed to it. Bizarrly on windows its works fine, with the same code
Doc Flabby - Sun Jan 20, 2008 9:55 am
Post subject:
Code: Show/Hide int do_dec(EncData *ed, byte *data, int len) { //packet to small if(len < 13) return 0; //my key holds key byte *mykey = (byte*)ed->key; //display IV if its there #ifdef CFG_LOG_CONNECTION lm->Log(L_DRIVEL, "<enc_ss2> CLIENTKEY"); dump_pk(mykey,16); lm->Log(L_DRIVEL, "<enc_ss2> END KEY"); #endif //get IV byte *IV = data; byte *Content = data+8; byte* moreData = amalloc(len-8); memcpy(moreData, Content, len-8); //init and clear decryption key byte deckey[24]; memset(&deckey[0], 0, 24); byte* deckeypos = &deckey[0]; memcpy (deckeypos, mykey, 16); memcpy (deckeypos+16, IV, 8); #ifdef CFG_LOG_CONNECTION lm->Log(L_DRIVEL, "<enc_ss2> IV"); dump_pk(deckeypos,24); lm->Log(L_DRIVEL, "<enc_ss2> END IV"); #endif struct MD5Context ctx; byte out[16]; MD5Init(&ctx); MD5Update(&ctx, deckeypos, 24); MD5Final(out, &ctx); #ifdef CFG_LOG_CONNECTION lm->Log(L_DRIVEL, "<enc_ss2> MD5"); dump_pk(&out[0],16); lm->Log(L_DRIVEL, "<enc_ss2> END MD5"); #endif //out contains the sessionkey #ifdef CFG_LOG_CONNECTION lm->Log(L_DRIVEL, "<enc_ss2> DATA TO DECRYPT"); dump_pk(moreData,len-8); lm->Log(L_DRIVEL, "<enc_ss2> END DATA TO DECRYPT"); #endif rc4_skip((byte*)&out, 16, 1024,moreData,len-8); memcpy(data,moreData+4,len-12); afree(moreData); #ifdef CFG_LOG_CONNECTION lm->Log(L_DRIVEL, "<enc_ss2> DECRYPTED DATA"); dump_pk(data,len-12); lm->Log(L_DRIVEL, "<enc_ss2> END DECRYPTED DATA"); #endif return len-12; } int Decrypt(Player *p, byte *data, int len) { #ifdef CFG_LOG_CONNECTION lm->Log(L_DRIVEL, "<enc_ss2> Decrypt %i",len); #endif EncData *ed, **p_ed = PPDATA(p, enckey); pthread_mutex_lock(&mtx); ed = *p_ed; pthread_mutex_unlock(&mtx); return ed ? do_dec(ed, data, len) : len; } |
I've narrowed down the problem. do_dec(ed, data, len) is not being called. However it should be as I have unwravelled the line into separate if statements and can confirm ed does exist so it should be called.
k0zy - Sun Jan 20, 2008 10:22 am
Post subject:
Maybe some other thread manipulates "ed" so it equals a value interpreted as false?
Dr Brain - Sun Jan 20, 2008 10:25 am
Post subject:
Set a breakpoint in the calling function and step it through.
Doc Flabby - Sun Jan 20, 2008 10:37 am
Post subject:
I located the problem
Code: Show/Hide int do_dec(EncData *ed, byte *data, int len) |
needed to be
Code: Show/Hide local int do_dec(EncData *ed, byte *data, int len) |
works like a charm now. Still don't understand what the difference between the two is...but nevermind...
Mine GO BOOM - Sun Jan 20, 2008 3:40 pm
Post subject:
local is an alias for static in ASSS. When you make a function static in C, it is known only in that file, no other. It helps keep the namespace clean.
GCC compiler attempted to find the best do_dec when compiling that file, and it found correct one. Windows's compiler attempted to find the best one, and it sounds like it found the vie encryption's do_dec, which is not local. It failed the tests in there, and returned nothing.
So yes, this is a bug in ASSS where the vie encryption's file is filling the namespace with functions it shouldn't be. I'd recommend reporting it as a bug.
k0zy - Sun Jan 20, 2008 3:54 pm
Post subject: