WCG restart delayed...until...13May 2023

[Jason Jung]

Haven't had any transient errors in a few days. OPNG is coming in short bursts like it did under IBM. Downloads are rather slow though. 20-30 KB/s verses over 300 KB/s a week ago. I'm wondering if it's caused by a network bandwidth limitation, now that they can handle more connections, or if it's caused by a higher load on the download servers.

 

[Vester]

It is going well enough that I have quit Folding@home. Some of my stats show updated information and I am hopeful that the total runtime will update soon. I have had plenty of OPNG tasks today. There is currently no ARP work, but that project is not one of my favorites.
Edit: Downloads/uploads and website rendering are slow again just hours later. I have run out of OPNG tasks. Some members report stalled ARP tasks.

 

[Vester]

A batch of OPNG just downloaded without delays.

 

[Jason Jung]

Got an e-mail from WCG with an update.

Dear Jason Jung,

We cannot thank you enough for your dedication to science and your support of the Grid during the transition from IBM.

Finally, with a functioning infrastructure and critical issues resolved we are ready to reboot the World Community Grid!

Your ongoing support and feedback during the transition from IBM has been invaluable to the scientists who rely on us. Together we improved the Grid’s functionality and efficiency by targeting our limited technical resources, and while we certainly encountered more obstacles and challenges than we anticipated we are here now because of your patience and persistence.

There is work that remains to be done. In particular, while we were able to restore the My Contribution page functionality and you may have noticed that results over the past 2 days are now reflected - we must now carefully iterate through a modified version of the stats update procedure to add back each day that was missed. The results tab of the My Contribution page does reflect accurately the validation status and assigned credit of your workunits.

When complete stats are available we will begin a grace period for streaks of one month, extend all streaks that were active before the transition, and finally restore the normal cacluation of streaks when the grace period ends.

Finally, we are preparing a well deserved Badge of Honor for all the volunteers who submitted a valid result during the transition and testing phase, yourself included. We are also preparing yet another badge for all citizen scientists who join or return to the grid before the New Year.

Our research partners - the ARP, HSTB, MCM, OPN1 and SCC research teams - would like to extend their sincere thanks for seeing them through this crisis. As scientists ourselves at the Jurisica Lab and also one of the scientific teams of the WCG, we are proud to count you among our colleagues in science and look forward to working with you as we expand WCG operations. While returning and maintaining the full capacity of the Grid is our mandate, we will now be preparing to onboard new projects.

The World Community Grid remains steadfast and unchanged in our vision of a healthier world. Our mission is to accelerate science by creating a supercomputer empowered by a global community of volunteers. WCG supports open-source and open-data research while providing scientists with a computing platform that allows them to answer the world’s most pressing questions.

Thank you for your contribution to WCG and enabling seemingly impossible scientific research to come to life,

WCG Team at Krembil Research Institute, UHN

 

[Vester]

I hope we are on the verge of major progress. I am down to about six tasks in progress and the downloading tasks shown in transfers tab have not made any progress in the past 18 hours or so. The website's main page just became available again.

Edit: Just a few minutes later and my downloads are making progress at about 1.2 to 2.4 Mbps.
2nd Edit: Download speeds decreased progressively. I am getting no downloads hours later. When I go to the WCG website, I get:

System error​

World Community Grid is currently experiencing an unexpected error. Please check Facebook or Twitter for more information.

 

[Jason Jung]

Seems they came into the office and started working on fixing stuff. It was going so smoothly for almost a week and then they sent out that e-mail saying they have officially restarted...at the start of a weekend...

 

[Vester]

Cyclops Advanced Cruncher Joined: Jun 13, 2022 Post Count: 79 Status: Recently Active

2022-10-03 (System error explanation)​ Hi everyone, As you may have noticed, the database storing user data for the WCG website and forums went down yesterday morning. The database is back online now and users are again able to login to the website and forums. We have identified the root cause of this problem, and put steps in place to prevent this from happening in the future. All data is intact, the BOINC backend was not affected, and we will be able to reflect the workunits sent in yesterday while the website was down in the My Contribution page shortly. We will also be able to include the backlog of results from the testing phase, which we are working to include in the various My Contribution widgets retroactively. Apologies for the inconvenience this must have caused. WCG Team at Krembil Research Institute, UHN

 

[Vester]

It is still down, and I am discouraged. It could be worse. I could be responsible for WCG's data center.

 

[Jason Jung]

World Community Grid - View Thread - 2022-10-04 Update (Thank You Email clarification)

www.worldcommunitygrid.org

Hi everyone,

We wanted to address some confusion that we have seen regarding the recent “World Community Grid is officially restarting BOINC” email. The point of the email was to express gratitude towards our community for continuing supporting WCG during the transition period, rather than announcing that the restart already happened. We are actively working on the many remaining issues, including the contribution page and calculation of the streaks (as noted in the email), download and http errors that you have been reported on the forums or our social media platforms. We specifically sent the email to the active volunteers, as they are helping us resolve the remaining challenges, and separately to the volunteers that supported WCG over the last 3 years, to start increasing the number of devices. We have not announced the restart to the general public yet - as we need to test and resolve issues such as the database crash on Sunday. We will announce the full restart once the platform is in a stable state with minimal to no errors - at that time on the web site and social media.

Once again, thank you for your support, patience and understanding.

WCG team at Krembil Research Institute

 

[Vester]

WCG tweeted 8 hours ago. Brief has been redefined.

World Community Grid

@WCGrid


The WCG website is down for a brief maintenance check, we will notify you once it has been finished and the website is back online.

 

[Jason Jung]

WCG tweeted 8 hours ago. Brief has been redefined.

That actually replied about a hour and a half after: "The website should be fixed now, thank you for your patience."

https://twitter.com/i/web/status/1578430435519963136

I wish they would just plain make a new tweets for everything so it's all in a nice chronological line.

 

[Vester]

Downloads are much better today. The website is very slow. My queue is staying full without intervention.

 

[Vester]

All is better today. I am looking forward to the statistics being brought up to date. I have had MCM, OPN1, and OPNG tasks in the past 24 hours.

 

[Vester]

Cyclops update:

Hi everyone,

We are happy to announce that we have been able to increase the number of WUs available to volunteers. Global Stats updates are running normally and My Contributions page dashboard has been updated daily since the Thank You emails were sent. They are available for most users now, and we are resolving the last issues that will bring this to all volunteers. We continue working on updating all the stats and displaying forum streaks on the website, but they are stored in the database and reflected in the results tab of the My Contribution page.

Once again, a huge thank you to everyone for supporting WCG at this stage, submitting bugs and helping other volunteers in the forums. It is great to see an increased flow of results back to scientific partners. It is exciting to see that run time days for the preceding four weeks reached 1,160,151, and almost half of it (528,805) was achieved in the last week alone. Thank you.

Unfortunately, as the workload increased, we have encountered several system errors over the past two weeks. While we thought we knew the root cause and how to prevent the error in the future, we did not uncover the full complexity of what caused the error. Although we are closer to understanding the main cause, we continue to collect metrics during these events so that we can resolve it fully. Once we find a permanent fix for this issue we will provide more details in an update.

For any questions about this announcement, please comment in this thread.

 

[Nick Name]

I noticed I hadn't gotten any work for a couple weeks, maybe longer. I started crunching TN-Grid again and thought maybe project balance had something to do with it. Checking the log showed some odd messages about work not being available except for GPU apps. I didn't think that was correct so I tried a project reset. That seems to have fixed the problem for now.

 

[Jason Jung]

Added another BOINC client and noticed it showed up under Device Management but some clients I added after the move still were not there. Detached and reattached the project and they are now all there and can be assigned different profiles.

 

[Vester]

Stalled downloads are causing long periods of GPU inactivity on my computer. I am running OPNG only at WCG and the CPU is busy on Rosetta@home.

 

[Vester]

WCG forum response: "Error 500: javax.servlet.ServletException: net.myvietnam.mvncore.exception.DatabaseException: Error executing SQL in MVNForumPermissionWebHelper.getPermissionsForGroupGuest."

 

[Vester]

The forum is back after 18 hours and tasks are downloaded briskly.

 

[Nick Name]

According to yesterday's Twitter there's a site problem that may still be unresolved. It looks like it's only affecting website access, not BOINC workflows.