http.sh: Scheduled Web Requests

In November 2008, a customer in England wanted to run a free online service that called web pages and scripts on a schedule. His vision was a cross between an uptime monitor and a cron schedule for web endpoints: you gave http.sh (archive) a page or script and a schedule, and it requested that page on time and reported how the request went. He wanted the scheduler written as a multithreaded Linux daemon in C. I wrote the daemon and delivered it in mid-December, then answered his setup and data questions through January 2009. I didn’t build the website.

What I Did

• A multithreaded scheduler. The daemon ran every account’s schedules on time, so site owners didn’t need access to cron or Task Scheduler on their own servers. Schedules ran every 1, 5, 10, 15 or 30 minutes; every 1, 2, 4, 6 or 12 hours; daily, or on a day of the week; or on the 1st or 15th of the month.
• GET and POST requests, with variables. A request could pass fields to the script it called, such as a secret key, so a search engine or a stray visitor hitting the same URL wouldn’t set it off.
• HTTP and HTTPS.
• A report on each request: the resolved IP address, header size, header download time and speed, and the HTTP status code. That report is what made it an uptime monitor as well as a scheduler.
• An offer to open-source it. The customer wanted the service to be free for everyone, so I offered that the code could be open-sourced. He never did, but his FAQ asked anyone interested in the code to let him know.

The daemon was written in C, with MySQL behind it.

Where It Went

http.sh made more than 100 million scheduled requests and stored their results. At one point, 800 endpoints were configured. It stayed in production for about 8 years, and by November 2016 it had been taken offline.