Collatz Optimization

[Nick Name]

If you're crunching Collatz make sure you add and apply specific parameters in the .config file. This will greatly speed up your crunching. This is public knowledge, there's a thread on the Collatz forum about it.

The .config file should be in your BOINC data folder, in the Collatz project folder. The location of the data folder depends on where it was placed by the BOINC installation. For Windows users I think that's usually in the Programs folder inside the BOINC program folder don't know what I was thinking there, the normal location is C:\ProgramData. I don't remember where the typical Linux location is. The file should be named something like collatz_sieve_1.30_windows_x86_64__opencl_ati_gpu.config or
collatz_sieve_1.30_windows_x86_64__opencl_nvidia_gpu.config depending on your GPU. If you have a 32-bit operating system your file should read x86_32. Years ago 64-bit systems got both a 32-bit and 64-bit app. If you have both, you'll need to edit the .config file for each app. Right now it looks like you should only get the one or the other.

The following is for Windows although the process will be similar for Linux.

1. Locate the BOINC Data -> projects -> Collatz folder.
2. Open the file with a plain text editor like Notepad, Nano, Vi, etc.
3. Enter the values and save. You can find some suggested values here.
My current values are as follows:

RX 480

verbose=1
threads=8
kernels_per_reduction=48
sieve_size=26
lut_size=16
sleep=0
cache_sieve=1

This gives me run times just over 17 minutes.

1080 Ti

verbose=1
kernels_per_reduction=48
sleep=0
threads=8
lut_size=17
sieve_size=30
cache_sieve=1

This gives me run times just over 4 minutes. If you have a CPU newer than the Intel 1366 socket and a motherboard supporting PCIe 3.0 you should do even better.

If the .config files aren't there, you can create them the same way. Just make sure you name them exactly as written above including the .config extension. For Windows users, make sure you save the file as All Files and not a text file. Be sure in either case that you are saving the file to the Collatz project folder.

You don't need to restart BOINC, the Collatz app will pick up the values on the next task.

Some comments from Slicker:
https://boinc.thesonntags.com/collatz/forum_thread.php?id=8&postid=124

sleep=0 any setting other than 0 will likely slow things down as it literally sleeps after submitting the kernel to the GPU.

cache_sieve=1 any setting other than 1 will add several seconds to the run time as it will re-create the sieve for each WU run rather than re-using it

reduce_cpu=0 This does nothing in the current version.

https://boinc.thesonntags.com/collatz/forum_thread.php?id=8&postid=286

FYI, bumping up the lut size on GPUs can slow down the processing. The goal is to find the lut size and cache that fit within the GPUs cached RAM. Anything larger is slower and anything smaller is slower.

https://boinc.thesonntags.com/collatz/forum_thread.php?id=8&postid=368

One way to check the speed on various settings without having to run the entire WU is to:

1. Copy the app, to a temp folder.
2. Copy the collatz config file to the temp folder but rename it to collatz.config
3. Copy a collatz WU file to the temp folder and rename it to in.txt
4. Run the WU for 15 minutes.
5. copy stderr.txt to stderr_test_N.txt changing N to a new number each time
6. delete the boinc_lockfile
7. delete the out.txt (probably won't exist unless the WU finished)
8. delete the checkpoint.txt file
9. delete the stderr.txt file
10. edit the config and try new settings
11. go back to step 4
12. compare the new stderr to the previous one and see which reports numbers in less time e.g. 1234567890 - 123 steps @ 1:03 vs 1234567890 - 123 steps @ 0:57

For GPU apps, you will also need to have an init_data.xml file in the temp folder to tell it which GPU type and number to use. You can copy one from https://github.com/BOINC/boinc/tree/master/samples/openclapp/INIT_DATA test files

This should speed up crunching but also push the card harder. If you're using the computer while crunching and things are laggy, try reducing the sieve_size, lut_size and kernels_per_reduction.

 

[Nick Name]

I recently worked with a teammate on this. Before implementing the config settings, credit was around 37k times on a 1080 Ti and run times were around 770 seconds. This gives a credit/second ratio of 48. Now, run times are under 300 seconds (I'll guess a rough average of 280 seconds) and credit is somewhat lower, I'll put it around 28k. However, this gives a credit/seconds ratio of 100. As you can see, the default Collatz values (no config settings) are very conservative. You can see a great benefit even if you aren't running newer high end GPUs.

 

[Scott]

It has been a few years since I crunched. I do not remember how to do a config file. I know nothing about coding. I do remember using notepad to make the files. Can anyone point me to a config files for dummies explanation? I looked at the thread Nick posted. It helps, but how to actually create the file and name it correctly is what I still am not sure of. I have this to work with, but do not know how to put it in a useable file or if it needs to be in 2 separate files and what type. I had all this crap saved, but it has been lost over the years.

verbose=1
kernels_per_reduction=48
threads=9
lut_size=17
sieve_size=30
cache_sieve=1
sleep=1

<app_config>
<app>
<name>period_search</name>
<max_concurrent>4</max_concurrent>
<gpu_versions>
<gpu_usage>0.50</gpu_usage>
<cpu_usage>0.25</cpu_usage>
</gpu_versions>
</app>
<app>
<name>collatz_sieve</name>
<max_concurrent>4</max_concurrent>
<gpu_versions>
<gpu_usage>0.50</gpu_usage>
<cpu_usage>0.25</cpu_usage>
</gpu_versions>
</app>
</app_config>

 

[Jason Jung]

On Windows make sure you have hidden items and file name extensions visible. All the project folders are in C:\ProgramData\BOINC\projects by default. Open up the Collatz folder and create a new text document and rename it app_config.xml Then edit it:

<app_config>
<app>
<name>collatz_sieve</name>
<max_concurrent>4</max_concurrent>
<gpu_versions>
<gpu_usage>0.50</gpu_usage>
<cpu_usage>0.25</cpu_usage>
</gpu_versions>
</app>
</app_config>

In that same folder create your collatz_sieve_whatever_version_matches_the_exe.config with:

verbose=1
kernels_per_reduction=48
threads=9
lut_size=17
sieve_size=30
cache_sieve=1
sleep=1

 

[Nick Name]

Make sure you're saving it as type All Files and not a text file, which is the default in Notepad.

I would also start with running just one task at a time so you can get a baseline for your runtimes, then increment up. Collatz is pretty taxing and 4 concurrent will probably lead to reduced production.

 

[Scott]

I tried the config file and every task failed so I deleted it. The Nvidia card does a task in 6 min. I think it could be faster since it is not fully loaded, but I don't want a big list of failed tasks.

 

[Nick Name]

It may be that the VRAM filled up if you were running 4 at once. I don't think anyone should run more than 2 unless you're using something like this.

NVIDIA A100 GPUs Power the Modern Data Center

The fastest data center platform for AI and HPC.

www.nvidia.com

It can take a good bit of time to determine the best settings because the credit varies per task. This was working well for me on a 2080 Ti.

verbose=1 kernels_per_reduction=48 sleep=0 threads=8 lut_size=18 sieve_size=30 cache_sieve=1

 

[Scott]

No, that was only ruining 1 task. It just screwed it all up

 

[Scott]

It may be that the VRAM filled up if you were running 4 at once. I don't think anyone should run more than 2 unless you're using something like this.

NVIDIA A100 GPUs Power the Modern Data Center

The fastest data center platform for AI and HPC.

www.nvidia.com

It can take a good bit of time to determine the best settings because the credit varies per task. This was working well for me on a 2080 Ti.

verbose=1 kernels_per_reduction=48 sleep=0 threads=8 lut_size=18 sieve_size=30 cache_sieve=1

I tried those settings and it worked and cut 4 min off of each task! Went from 6 min 23 sec, to 2 min 23 sec. Wow. It is so close to what I had, but did the trick. My first settings I tried-

verbose=1
kernels_per_reduction=48
threads=9
lut_size=17
sieve_size=30
cache_sieve=1
sleep=1
Any suggestions for my AMD 6500XT? It is taking 22 min per task. It is not even close to a powerful GPU, but I bet it can do better I may send it to MW otherwise. It does well there.

 

[Nick Name]

I tried those settings and it worked and cut 4 min off of each task! Went from 6 min 23 sec, to 2 min 23 sec. Wow. It is so close to what I had, but did the trick. My first settings I tried-

Make sure the credit isn't dropping along with the time. I don't recall the exact figures but when I was testing settings I saw lower run times but also reduced credit. Since the credit isn't fixed the variance makes it a bit tricky.

I don't have specific suggestions for that 6500XT. I would start with the base settings and increase from there. I think the lut_size and sieve_size had the greatest effect.

 

[Scott]

What are the "base settings"? I can't find them anywhere.

 

[Scott]

Ok I tried these for the AMD. It cut the tasks from around 1400 sec to just over 700 sec. Almost in half. I am moving into 1st for RAC


verbose=1
kernels_per_reduction=48
threads=8
lut_size=17
sleep=1
reduce_cpu=0
sieve_size=30

 

[Nick Name]

What are the "base settings"? I can't find them anywhere.

Frankly, it's been so long since I messed with this that I don't remember. My thought was there should be a default .config file created when you attach to the project, but that may not be true. Anyway, it's great to see you making progress and climbing the ranks.

 

[Scott]

Actually there is nothing in the config files. It is just blank until I added the settings. Took #1 spot for RAC. It will take a while to hit #1 in total credit