It is currently 10 Sep 2010, 03:38

All times are UTC + 1 hour [ DST ]




 Page 5 of 14 [ 197 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 14  Next
Author Message
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 22 Nov 2008, 14:36 
Developer

Joined: 30 Mar 2008, 15:37
Posts: 847
i guess the check for nRead==8 has been placed there for a good reason (maybe crashing because of trying to copy data while there were fewer bytes).

but i was already trying some other stuff, but i'll continue later.


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 26 Nov 2008, 22:53 
Developer

Joined: 30 Mar 2008, 15:37
Posts: 847
well, i could read everything into a 'chain' at once, not using nIndexS and nIndexE, then the chain is masked and shifted when those values are needed, like this:

uint64 nIndexS = pChain[nIndexToVerify].nIndexS & 0x0000FFFFFFFFFFFF; // for first 6 bytes
unsigned short nIndexE = pChain[nIndexToVerify].nIndexS >> 48; // for last 2 bytes


it might be a little faster in the end, but i don't think it's making the code more readable. And everything will probably change on the next version anyway, as there might be checkpoints in use, and PowerBlade is working on RTI2 format. We'll see about it later ;)


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:09 
Developer

Joined: 30 Mar 2008, 15:37
Posts: 847
Here is version 0.4 WITH source ! :mrgreen:

Changes:

* Memory usage down to about 50% (tnx sc00bz for pointing out the inefficiency, tnx jci for helping with the code)
* Works under linux!
* Speed up: implemented MD4 reference implementation, not using OpenSSL for NTLM anymore (tnx Bitweasil for supplying 98% of the code :))
* Speed up: using OpenSSL a lot faster (tnx jci for pointing that out)
* Probably some more small things i forgot

I left some notes in the source, crediting bitweasil and jci, hope you don't mind :P

Compile under linux (can't be bothered to verify right now :roll: ):
g++ *.cpp -lssl -pthreads -O3 -o rcracki_mt

pausing and skipping lm2ntlm doesn't work under linux, couldn't be bothered for now.

The Windows binary now comes with 2 dlls: one for openssl, one for posix threads.


Attachments:
File comment: Source (now with all files :P)
rcracki_mt_0.4_src.zip [45.57 KiB]
Downloaded 205 times
File comment: Windows binary
rcracki_mt_0.4_win32.zip [599.5 KiB]
Downloaded 252 times


Last edited by neinbrucke on 27 Nov 2008, 00:57, edited 1 time in total.
Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:14 
Brute Force

Joined: 05 Nov 2007, 01:55
Posts: 103
neinbrucke wrote:
well, i could read everything into a 'chain' at once, not using nIndexS and nIndexE, then the chain is masked and shifted when those values are needed, like this:

uint64 nIndexS = pChain[nIndexToVerify].nIndexS & 0x0000FFFFFFFFFFFF; // for first 6 bytes
unsigned short nIndexE = pChain[nIndexToVerify].nIndexS >> 48; // for last 2 bytes


it might be a little faster in the end, but i don't think it's making the code more readable. And everything will probably change on the next version anyway, as there might be checkpoints in use, and PowerBlade is working on RTI2 format. We'll see about it later ;)


About readability, you could:
1) use a union, with a struct for the nIndexEnd, with 3 "filling" so the real value starts at byte 6.
2) use 2 simple defines
#define getChainStart(nIndex)  (pChain[nIndex].nIndexS  & 0x0000FFFFFFFFFFFF )
#define getChainEnd(nIndex)    (pChain[nIndex].nIndexE.value)

// ...

uint64 nIndexS = getChainStart(nIndexToVerify);
unsigned short nIndexE = getChainEnd(nIndexToVerify);


You might be right about the other points though, but meanwhile - IMHO - if this works, uses 50% less RAM, and is at least as fast (if not faster), then why not ?


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:19 
Developer

Joined: 30 Mar 2008, 15:37
Posts: 847
i used an array instead of a struct before, but i somehow found it a little lame to use :P
(i have no clue about programming 'decency' and such... so usually i really don't care :))

i'll leave it with version 0.4 as it is right now (memory down to 50% and i didn't notice any drop in speed (reading from disk is way slower anyway)), and we'll try to make it nice and clean when the new changes are introduces.


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:24 
Rainbow Table

Joined: 18 Oct 2007, 19:30
Posts: 323
small & quick reply :
_ i had an error, i just replaced the libeay32.dll by "my own" libeay32.dll and it works fine now (don't ask me what was wrong, same dll version)
_ do you think it's possible to use normal/perfect rainbow tables with it (.rt) ?

Thx in advance :)
(and good work)


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:27 
Developer

Joined: 30 Mar 2008, 15:37
Posts: 847
if more people have that error, i'll swap the dll with the one that came with 'original' rcracki

why would you want to use normal rt's with rcracki? (and to answer your question, yes, should be possible, but don't think it will be done)


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:39 
Rainbow Table

Joined: 18 Oct 2007, 19:30
Posts: 323
because the original rcrack isn't multithreaded
because i have rti and rt files in the same directory
because it's easier to have one exe wich works with both than 2 exe (one for rt and another one for rti)
because making it "compatible" could be usefull
because upgrading one binary is easier than 2 (routines, etc)


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:46 
Developer

Joined: 30 Mar 2008, 15:37
Posts: 847
that's a nice list ;)

so i guess you still have .rt tables which don't have a similar (or better) .rti variant ?


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:50 
Brute Force

Joined: 05 Nov 2007, 01:55
Posts: 103
Having a few issues building on Linux, but mainly the file ChainWalkContext.cpp that's missing :?


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:52 
Brute Force

Joined: 05 Nov 2007, 01:55
Posts: 103
pointp wrote:
because i have rti and rt files in the same directory


I think what should use there is "rto2rti" :D oh, wait... :roll:


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 00:56 
Developer

Joined: 30 Mar 2008, 15:37
Posts: 847
you don't want to rto2rti with every old rto... the current rto2rti does indexing, but also perfecting... so you'd reduce success rate.

and ehm, oops about that file :D

[edit]i uploaded a new src zip
btw, what other errors do you get? i only tested on an ubuntu 8.04 32bit


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 01:12 
Brute Force

Joined: 05 Nov 2007, 01:55
Posts: 103
neinbrucke wrote:
you don't want to rto2rti with every old rto... the current rto2rti does indexing, but also perfecting... so you'd reduce success rate.

and ehm, oops about that file :D

[edit]i uploaded a new src zip
btw, what other errors do you get? i only tested on an ubuntu 8.04 32bit


I'm building with gcc 4.3+ (Ubuntu 8.10)

my changes:

in Public.h, "u_int64_t" is not defined (throws a LOT of errors everywhere):
#ifndef u_int64_t
   #define u_int64_t unsigned long long
#endif


in hash_md4.h "__int32" is not defined:
#ifndef __int32
   #define __int32 long
#endif


a lot of undefined references to memset, memcpy, etc
in public.h:
#include <stdlib.h>
#include <string.h>


Also, in CHashRoutine.cpp, a lot of hashes could use the "MD?_NEW" functions instead of openssl, and drop "<openssl/md?.h"> altogether

Edit:
To build on Linux use "-lpthread" and not "-pthreads"


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 04:55 
Guesser

Joined: 26 Apr 2008, 12:27
Posts: 40
Build it under debian etch with the changes of jci successful. Thanks a lot for your work.
Regards goody


Offline
 Profile  
 
 Post subject: Re: An unofficial multithreaded rcracki!
PostPosted: 27 Nov 2008, 06:40 
Brute Force

Joined: 05 Nov 2007, 01:55
Posts: 103
I am benchmarking the MD4_NEW ... with mixed results:

with lengths aligned to 2 bytes performance is good, but for odd lengths (5,7,etc) its... slow... :(

length 4, 6 is about the same:
Number of threads:  1
Number of hashes:   200M
Hashes per thread:  200000000
Running benchmarks, please wait...

*** Algorithm:    md4_new
Time ellapsed:      21.201 seconds
Hash performance:   9.434M/sec total

*** Algorithm:    fast_md4
Time ellapsed:      19.487 seconds
Hash performance:   10.263M/sec total


length 5:
*** Algorithm:    md4_new
Time ellapsed:      34.931 seconds
Hash performance:   5.726M/sec total

*** Algorithm:    fast_md4
Time ellapsed:      18.648 seconds
Hash performance:   10.725M/sec total


length 7, with 2 threads:
*** Algorithm:    md4_new
Time ellapsed:      18.469 seconds
Hash performance:   10.829M/sec total

*** Algorithm:    fast_md4
Time ellapsed:      10.020 seconds
Hash performance:   19.959M/sec total


If you want to, have a look at these:

MD4:
fast_md4.c fast_md4.h

MD5 (slightly faster than MD5_NEW):
fast_md5.c fast_md5.h


Edit:
I've made a few small modifications to your routine, it's now almost twice as fast on odd lengths, and a tad faster on evens too. :D
It's also faster than my own version :oops: , (maybe due to smaller code / not supporting any give length?)
(by the way, it crashed on any length > 31, i added a check so it just returns incorrect hashes )

length 7, 2 threads:
*** Algorithm:    md4_new
Time ellapsed:      9.810 seconds
Hash performance:   20.388M/sec total

*** Algorithm:    fast_md4
Time ellapsed:      10.289 seconds
Hash performance:   19.438M/sec total


Offline
 Profile  
 
Display posts from previous:  Sort by  
 Page 5 of 14 [ 197 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 14  Next

All times are UTC + 1 hour [ DST ]


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: