Multi tool use

C# - Improve performace when searching

i'm having a list of 15000000 username on txt file & i wrote a method to create brain wallet out of it check if any address contain with a list of 600 address. It's pretty much like this

private static List<string> userList = new List<string>(File.ReadAllLines(@"C:UsersErikDesktopInfernoUser-workspace-db.txt"));
private static List<string> enterpriseUserList = new List<string>(File.ReadAllLines(@"C:UsersErikDesktopInfernoEnterpriseUser-local-db.txt"));
foreach (var i in userList)
{
userid = ToAddress(i);
if (enterpriseUserList.Contains(userid))
Console.WriteLine(i,userid);
{
private string ToAddress(string username)
{
string bitcoinAddress = BitcoinAddress.GetBitcoinAdressEncodedStringFromPublicKey(new PrivateKey(Globals.ProdDumpKeyVersion, new SHA256Managed().ComputeHash(UTF8Encoding.UTF8.GetBytes(username), 0, UTF8Encoding.UTF8.GetBytes(username).Length), false).PublicKey);
}


ToAddrsess method hash username into SHA256 string, get its public key & convert it into address like this:

15hDBtLpQfcbrrAFupWjgN5ieHeEBd8mbu


This code is ass, run really slow, handle about 200 line of data per second. So i try to improve it using multithreading

private static void CheckAddress(string username)
{
var userid = ToAddress(username);
if (enterpriseUserList.Contains(userid))
{
Console.WriteLine(i,userid);
}
}
private static void Parallel()
{
List<string> items = new List<string>(File.ReadLines(@"C:UsersErikDesktopInfernoUser-workspace-db.txt"));
ParallelOptions check = new ParallelOptions() { MaxDegreeOfParallelism = 100 };
Parallel.ForEach<string>(items, check, line =>
{
CheckAddress(line);
});
}


It didn't help much. Can anybody suggest how to improvise this? compare to vanitygen run on CPU which can handle 4-500k address per second. How can it make such a big difference?

Contains

3 Answers
3

You can try to use Dictionary with key=userid, to prevent search by list each iteration

var dict = new ConcurrentDictionary<string, string>(100, userList.Count);

userList.AsParallel().ForAll(item =>
{
dict.AddOrUpdate(ToAddress(item), item, (key,value)=>{return value;});
});

enterpriseUserList.AsParallel().ForAll(x =>
{
if (dict.ContainsKey(x))
{ Console.WriteLine(dict[x]); }
});


.AsParallel()

.AsParallel()

Console.WriteLine

foreach

Console.WriteLine

Console.WriteLine(stringBuilder.ToString());

When looking for inefficiencies one of the major red flags is repeated function calls. You call GetBytes twice. Putting it into a separate variable and calling it once should help some what.

GetBytes

private string ToAddress(string username)
{
var userNameAsBytes = UTF8Encoding.UTF8.GetBytes(username);
string bitcoinAddress = BitcoinAddress.GetBitcoinAdressEncodedStringFromPublicKey(new PrivateKey(Globals.ProdDumpKeyVersion, new SHA256Managed().ComputeHash(userNameAsBytes, 0, userNameAsBytes.Length), false).PublicKey);
}


you can perform some operations here

List

HashSet

Contains

private static List<string> enterpriseUserList = new List<string>(File.ReadAllLines(@"C:UsersErikDesktopInfernoEnterpriseUser-local-db.txt"));

private static HashSet<string> enterpriseUserList = new HashSet<string>(File.ReadAllLines(@"C:UsersErikDesktopInfernoEnterpriseUser-local-db.txt"));

ParallelOptions check = new ParallelOptions() { MaxDegreeOfParallelism = 100 };

Parallel.ForEach

Partitioner.Create

maybe that's all I can advise you.

private static List<string> userList = new List<string>(File.ReadAllLines(@"C:UsersErikDesktopInfernoUser-workspace-db.txt"));
private static HashSet<string> enterpriseUserList = new HashSet<string>(File.ReadAllLines(@"C:UsersErikDesktopInfernoEnterpriseUser-local-db.txt"));

[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static void CheckAddress(int id,string username)
{
var userid = ToAddress(username);
if (enterpriseUserList.Contains(userid))
{
// todo
}
}


private static void Parallel()
{
var ranges = Partitioner.Create(0,userList.Count);
Parallel.ForEach(ranges ,(range)=>{
for(int i=range.Item1;i<range.Item2;i++){
CheckAddress(i,userList[i])
}}

}


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.