tnx all, could still use more for verification.

mastergamer, which language?
probably german, french, danish of something?

Windows XP PRO SP3 Dutch: 850
Windows 2003 Enterprise SP2 English: 437

Sorry - forgot to mention. It's English.

ok, i think i've got quite a nice characterset now, little problem... it contains 143 characters, NOT including the first 31 control characters.
So now i stripped all the bytes that are not so likely to be used for a password, those stupid blocks and lines from good old ascii art programs like ░▒ │╣║╗╝▀«»

Now i have a charset, containing about all relevant characters from the following OEM code pages (129 now):
(see http://www.microsoft.com/globaldev/reference/oem.mspx)

• 437 (US)
• 737 (Greek)
• 775 (Baltic)
• 850 (Multilingual Latin I)
• 852 (Latin II)
• 855 (Cyrillic)
• 857 (Turkish)
• 866 (Russian)
• 932 (Japanese Shift-JIS)

Partial: 865 — Nordic

I probably miss some characters from the following, less important code pages:
(also see http://en.wikipedia.org/wiki/Code_page)

# 860 — Portuguese
# 861 — Icelandic
# 863 — French Canadian
# 866 — Cyrillic
# 869 — Greek (not used as default code page)
• 874 (thai)

I couldn't get the following code pages active, not available on my systems:
• 720 (Arabic)
• 862 (Hebrew)

I'm gonna see if 720 and 862 have a lot of different mapping characters. Installing Arabic windows now

Also see http://www.microsoft.com/globaldev/reference/oslocversion.mspx for default code pages for different localized windows versions.
• 858 (Multilingual Latin I + Euro) (seems never to be used on a localized version... not by default at least, and none of you guys had it as default code page).

no missing hebrew, for arabic some 'undefined' characters missing (so no problem) and A4 = U+062C : ARABIC LETTER JEEM missing.... i'm not gonna care less

So here it is... it might become a bit of a problem to work with it... you might see a lot of question marks or blocks, but the original rcrack and rtgen works fine with it (the program reads the bytes, not the characters as you might see them):


i'd just like some comment on the reachability of this character set, i don't get it right with that excel sheet of scoobz
but keyspace with 129 characters, length 1-7 should be 2^49,09 .

errrr well
tell me what your plan is
10000x69999999 ? and how many tables? i think this charset might be a bit to big...

if you would use 20000x99999999 and 1000 tables you'd get 96,44% but youd also need 1,5TB.... PB has your server enough space for that ?

how did you calculate that? and is that for indexed perfect tables?

ehmm i took winrtgen and calculated it.
but your right indexing them would reduce the size to 50% havent thought about that...

and perfecting also lowers the size... we just need to calculate a lot

but perhaps this is just too big... maybe we should do charsets per codepage. i'll look into that tomorrow, i guess with 2 or 3 code page groups, we should be down to 100-120 characters per set.

That might be a wise idea.. Lets try calculate on the different options

I'm missing the German characters... Like Ä, Ö, Ü and ß.

Not sure though, if Windows converts them to A, O, U, etc.

no i think they are missing...

you can't tell by looking at the characters you see here... they even change when you set your browser to another character set.
it's about the bytes these characters resemble.

and depending on the code page your version of windows uses, windows maps a character to these bytes. i'll try to give an overview of the bytes and the characters per code page that windows maps lm hashes to

in short, german characters should be cracked

bah, i made a mistake

all above is still valid, although some hashes might still be missing, i'll explain later

ok, for a little explanation.
i thought i generated all 256 characters per OEM code page and mapped out the resulting LM hashes. But my mistake was that i generated the first 256 unicode characters. Although this is is useful character set, OEM code pages also contain characters from higher unicode characters.

So i started again from the start, taking in short these steps:

* Per code page, get a list of bytes and their respective unicode bytes (2 bytes). These are available for example from the following URL:
http://www.microsoft.com/globaldev/reference/oem/850.mspx
* Switch the default OEM code page in Windows and generate LM hashes for all 256 characters of that OEM code page
* Collect all the hashes and take the unique LM hashes and check which bytes crack these characters. These bytes will form the final charset that rcrack will need to crack the LM hashes.
* Generate all LM hashes for all 65536 unicode characters. I did this for cp850, and found that all characters indeed map to one of the characters in the code page (if they map at all). It is however important to get a full list of all the mapped characters, to be able to quickly bruteforce the matching NTLM hash.

So i did this for the following code pages (of these i consider 437 (US) and 850 (European) currently the most valuable for now):

• 437 (US)
• 720 (Arabic)
• 737 (Greek)
• 775 (Baltic)
• 850 (Multilingual Latin I)
• 852 (Latin II)
• 855 (Cyrillic)
• 857 (Turkish)
• 862 (Hebrew)
• 866 (Russian)
• 874 (Thai)
• 932 (Japanese Shift-JIS)
• 1258 (Vietnam)

In total, all the code pages make up for 230 unique LM hashes. This makes sense, as there are 256 possible input bytes, and in all the OEM code pages lowercase alpha gets converted to uppercase alpha, so you have max 256-26=230 possible input bytes. It is funny to see that you can have different LM hashes for equal characters in different OEM code pages It won't get you into trouble that easy, because nowadays most systems use NTLM for actual password verification anyway.

So this all seems nice, but 230 bytes and 1-7 in length is for now FAR too much for us to generate rainbow tables for. We can skip the first 32 (including 0x0000, no password) characters (control) characters for every code page, as these are not realistic to be used in passwords. So then we are left with a character set of 198 bytes, which is still far too large. It might however be interesting to generate 1-6 as this might give quite some success on cracking the second LM hash (and the first, for really short passwords ).

So what could we do to get useful character sets is split up per code page. But as i now mapped out all the characters per code page, there are still too many unique lmhashes per code page. For example (without the first 32):

* cp437: 171
* cp850: 167

Combined, these make up for 173 unique bytes. If we dump again the blocks and lines we are just as unrealistic to be used in a password, we leave away characters from these unicode blocks:

* Box Drawing (U+2500 - U+257F)
* Block Elements (U+2580 - U+259F)
* Geometric Shapes (U+25A0 - U+25FF)

Then we have these 123 bytes for cp437 (contains many box/block) and 139 bytes for cp850. This is probably still too much, although scoobz might shine his calculative light upon this
I calculate a keyspace of around 2^48.6 (might be possible?) for cp437 and 2^49.8 for cp850 (too much i gues).

We could strip away some more characters that are not likely to be used in passwords, so we are about to end up with just those characters that people use in real text, like characters with accents and such

* We could at least skip 0x007F (DELETE) and 0x00A0 (No Breaking Space).
* We could skip mathematical and technical characters like: ⌠⌡≈°∙√ⁿ unicode blocks: Mathematical Operators (U+2200 - U+22FF) and Miscellaneous Technical (U+2300 - U+23FF)
* We could manually pick 'useless' characters, i picked these:

* We could skip some currency symbols, but i think we're small enough now (would save a couple more).

So now i'm down to 96 bytes for cp437 and 108 for cp850. Combined they take up 114 bytes.

I would especially like some comments on the characters i manually removed, feel free to comment.

The numbers here could change a little, because some characters that are not in the specific OEM code page still map to these. So i need to verify those 65536 for all code pages and see if any other useful characters are removed because of this selection. Edit: I just did for cp850, no relevant characters were removed.

p.s. to see the above special characters correctly, you might need to set your browser to unicode/UTF-8
edit2: see picture to know for sure