distributedcomputing.info logo
Home   Forum    Wiki    Blog    News and Articles   Active Projects   Computing Platforms  
Upcoming Projects   Recently Completed Projects   Past Projects  
Tools   Development Platforms   Conferences   Parody and Fun Sites   Links   Politics
Teams   Books and Journals   FAQ   Link to Us

Active Distributed Computing Projects - Internet

These links take you to other project categories on this site:

Science   Life Sciences   Cryptography   Internet   Financial
Mathematics   Language   Art   Puzzles/Games   Miscellaneous   Distributed Human Projects
Collaborative Knowledge Bases   Charity

See the bottom of this page for a description of the icons on the page.

Project Information Project % Complete Major Supported Platforms
Gómez, Inc. Evaluate the performance of large websites to find bottlenecks with Gómez's distributed PEER client and its first project, peerReview. Some users will be paid up to US$45 per month for their contributions. Payments are made via PayPal or AliPay. The latest newsletter was published in Spring, 2009.

Note: peerReview does its work when a network connection is present. Modem users will notice that the client is not active while they are offline. A work unit is completed in 15 minutes, so it is possible for modem users to contribute useful work. The client supports some firewalls; it does not support users behind proxy servers.

The latest version of the PEER client is available as of April 22, 2009 for Windows. The client allows you to configure how much work your system contributes. PEER users' clients will update automatically.

ongoing paid project

Windows 32

NETI@home provides to network researchers "a wealth of end-to-end data that has been previously unavailable" in order to make the Internet faster and more reliable.

"NETI@home is an open-source software package that collects network performance statistics from end-systems. ... NETI@home is designed to run on end-user machines and will collect various statistics about Internet performance. These statistics will then be sent to a server at the Georgia Institute of Technology (Georgia Tech), where they will be collected and made publicly available. We believe that this tool will give researchers much needed data on the end-to-end performance of the Internet, as measured by end-users. Our basic approach is to sniff packets sent to and received by the host and infer performance metrics based on these observed packets. NETI@home users will be able to select a privacy level that will determine what types of data will be gathered, and what will not be reported. NETI@home is designed to be an unobtrusive software system that runs quietly in the background with little or no intervention by the user, and using few resources.

"NETI@home is written in C++ and uses the popular Ethereal network analyzer to sniff packets. NETI@home also uses the zlib compression library. ... NETI@home has been developed as a thesis project at Georgia Tech and has been designed in accordance with the CAIDA specifications on network statistics.

"NETI@home includes the NETIMap application, written in Java, to encourage the use of NETI@home. The NETIMap application, when run, will display a geographical map of the world. As Internet hosts are contacted (a website, for example), a dot is placed on that host's calculated coordinates. The coordinates are calculated using CAIDA's NetGeo database."

Warning for version 1.0: a user has reported that he installed NETI@home on a Win2000 SP4 machine, and after uninstalling it, he noticed there was a service running, trying to execute Neti@home every 20 minutes. He disabled the service, but was unable to remove the registry entries because there was always an "Error during delete." NETI@home personnel responded with the following information:

As a temporary solution, I would recommend using Control Panel->Administrative Tools->Services to stop the service and also to have the service disabled at startup. The strange thing is that the service code is located in "Program Files\Ethereal\neti" and if the program has been uninstalled, this folder should not exist or be empty. NETI@home runs as a service to avoid users having to manually start it and so that it can capture data when users are not logged in (just sitting at login prompt). To manually run NETI@home you can run the go.bat file located in "Program Files\Ethereal".

Version 2.0 of the client is available for Windows, Linux, Mac OSX and Solaris as of June 1, 2005.

ongoing Windows 32
Help DIMES (Distributed Internet MEasurements & Simulations) study the structure and topology of the Internet. DIMES is part of EVERGROW. "The DIMES agent performs Internet measurements such as TRACEROUTE and PING at a low rate, consuming at peak 1KB/S. The agent does not send any information about its host's activity/personal data, and sends only the results of its own measurements. Aside from giving a good feeling, running the DIMES agent will also provide you with maps of how the Internet looks from your home (currently) and will (in the future) provide you with a personalized 'Internet weather report' and other user-focused features."

DIMES passed 100 million measurements on June 16, 2005.

The client uses very little CPU and data bandwidth. It supports users behind firewalls. It requires you to have Java 1.4 or later installed on your system. Version 0.4.3 of the client is available for testing as of January 3, 2006. Version 0.4.2 of the client is available for Windows. Version 0.4.2b is available for Linux and Mac OSX. Source code for the client is available for download.

View VRML 3D maps of the Internet from the project's Community page.

Use the DIMES Internet Mapper (based on the Google Maps API) to visually browse the connections between the cities of the world, as measured by DIMES.

Join a discussion forum about this project.

29,404 networks and 204,204 links traced;
7,522,460,896 total measurements
Windows 32
Majestic-12 is a distributed World Wide Web search engine. The project's client software application "crawls" websites to see which sites have changed their content, and updates a master search index.

The project crawled 1,000,000,000 URLs by September 5, 2005. As of March 17, 2006, the project has 1 billion web pages indexed and searchable. The project discovered its 1 trillionth unique URL on October 12, 2009. The only other organization to publicly announce it has discovered 1 trillion URLs is Google.

Version 1.6.11 of the MJ12node client is available for Windows and Linux as of December 6, 2009. Windows users must have Microsoft .NET version 1.1 or 2.0 installed. Linux users must have Mono installed. An MJ12agent application is also available: it allows you to control an MJ12node client on a separate computer.

A Majestic12 search engine plug-in is available for the Firefox browser as of August 29, 2005.

Join a discussion forum about this project.

184,629,065,290 URLs indexed
Windows 32
Boitho is a general World Wide Web search engine. The project's client software application "crawls" website URLs to index their content, and creates thumbnail images of them for its master search index. This site is also available in Norwegian .

To participate in the project, download and install BoithoCrawler.exe, then run the BGui executable or use the Start menu: start -> All programs -> BoithoCrawler -> Start Boitho Crawler. When the application starts, move the "Run Nr of crawlers" slider to the right to start 1 or more web crawlers. The BoithoCrawler client is available for Windows.

Join a discussion forum about this project. See the latest news about the project in the Boitho blog.

397,263,160 URLs indexed
Windows 32
PeerFactor is a distributed computing and peer-to-peer file-sharing project. The site is also available in French . It uses its participants spare bandwidth and CPU cycles, and rewards the participants with game and music downloads. Participants receive credit for the results of the file-sharing and computing that their PeerFactor client completes, rather than the amount of time they make their computer available, so anyone, including modem-users, can participate. Participants will be able to redeem their credits for free downloads from commercial game and music download services. For now participation is only open to French citizens. An English version of the client will be available in late 2006, and participation will open to citizens of all countries when that version is available.

The project currently runs the following applications: peer-to-peer (P2P) sharing of legal promotional content (see some examples); improvement of search engine performance. In the future it will include other applications: benchmarking and monitoring of websites; distributed computing; rendering graphics; weather-related computing.

To participate in the project, download and run PeerFactor.exe. The PeerFactor client is currently only available for Windows. A Linux version should be available soon. The client is not configurable, to make it easier to use. The client runs at idle priority. The client can be used behind a firewall, but the firewall should authorize PeerFactor to use the maimum possible number of ports. The client currently only uses about 5% of available CPU to crawl and index websites for commercial search engines.

ongoing dialup-friendly

Windows 32

DepSpid Help DepSpid (the Dependency Spider) complete its two major goals:

1st: Build up a database containing the dependencies between individual web sites and groups of web sites.
2nd: Collect statistical data about the structure of the web.

All of the projects results will be made publically available.

As of June 22, 2008, no work units are available for phase-1. The project owners are developing phase-2 and will issue new work units when that phase is ready.

The project uses the BOINC computing platform to run various applications. See the BOINC platform information for the latest version of the BOINC client. Version 5.07 of the project's Spider software application is available for Windows as of December 17, 2006.

Join a discussion forum about this project.

waiting for phase-2 to begin;

Windows 32

Pingdom GIGRIB Help Pingdom GIGRIB measure uptime for websites. Participants in the project monitor the uptime of websites specified by other project participants, and can specify their own websites to monitor. The project's "website shows the uptime information for the thousands of websites that have been added by [its participants]. The information is public and searchable." The project is sponsored by Pingdom, which also runs a commercial, more fully-featured version of the website monitoring service using Pingdom's own servers. "GIGRIB stands for Green Is Good, Red Is Bad." See more information about the project and the project's FAQ.

To participate in the project, sign up for a free account, then download and run the project's client software application. Version 1.0.129 of the GIGRIB client is available for Windows as of December 11, 2006.

ongoing Windows 32
Grub, owned by Search Wikia Labs, is attempting to create a better search index than Google. The project is designed on "Four Organizing Principles (TCQP) - the future of Internet Search must be based on:
  1. Transparency - Openness in how the systems and algorithms operate, both in the form of open source licenses and open content + APIs.
  2. Community - Everyone is able to contribute in some way (as individuals or entire organizations), strong social and community focus.
  3. Quality - Significantly improve the relevancy and accuracy of search results and the searching experience.
  4. Privacy - Must be protected, do not store or transmit any identifying data."

The project, which began independently in 2001, was bought and ended by LookSmart on October 11, 2005. In its first incarnation it was "an open-source distributed Internet crawler." The client "crawled" websites to see which sites had changed their content, and updated a master search index in real-time. grub.org hoped to create and maintain the most comprehensive and up-to-date search index of the Internet ever, and provided update feeds of crawled sites to the public for free and to commercial search engines. LookSmart gave the following reason for ending the project:

Effective October 11, Grub.org, a project of LookSmart, will be shutting down its servers. LookSmart is focused on dedicating our internal resources to meet objectives that are core to our business strategy. As a result, certain elements that are no longer central to the business are being phased out.
Wikia bought the project from Looksmart in 2007 and is currently restarting it.

See the project's wiki.

waiting to begin Windows 32

The following icons may appear in the Supported Platforms section of the table:
dialup-friendlythis project is good for users with dialup Internet access
paid projectthis is a for-pay project
Windows 32this project runs on the Windows 32-bit platform
Linuxthis project runs on the Linux platform
MacOSthis project runs on the Mac OS platform
Solaristhis project runs on the Solaris platform