Learn Cloudflare, Nightmare and Twit

Cloudflare Workers let you write JavaScript Service Workers that run directly in Cloudflare's 200+ edge locations around the world.

With a Cloudflare Worker, you can modify your site's HTTP requests and responses, make parallel requests, and even reply directly from the edge.

Cloudflare Workers are written against a similar API to the W3C Service Workers standard. However, while Service Workers run inside the end user's browser, Cloudflare Workers run on Cloudflare's servers. This means that Cloudflare Workers can implement trusted code (which the client cannot tamper with), and will work no matter what browser the user is running.

In this course, Kristian Freeman will teach you how to build and deploy a Serverless API with Cloudflare Workers. By learning how to build your own serverless JSON APIs, you can effectively manage a highly-available backend for your projects, without needing to learn DevOps or manage services.

Cloudflare secures and ensures the reliability of your external-facing resources such as websites, APIs, and applications. It protects your internal resources such as behind-the-firewall applications, teams, and devices. And it is your platform for developing globally-scalable applications.

cloudflare

Build a Serverless API with Cloudflare Workers

Cloudflare Workers is a serverless application platform that enables you to create and deploy functions to Cloudflare's Edge Network.

Unlike other serverless providers that only have regional data centers, Cloudflare's Edge Network consists of hundreds of servers all over the world.

Cloudflare Workers is a powerful platform, and this course serves as a great introduction for what it can do.

Follow along with Kristian Freeman as you build a localization engine that renders data based on the Edge location nearest to the application's user.

After this course, you'll be ready to start experimenting with your own Cloudflare Workers projects. *Need an idea?* *[Check out the suggested Portfolio Project for this course.](https://egghead.io/projects/introduction-to-cloudflare-workers)*

Introduction to Cloudflare Workers

In this course we’ll learn how to create Twitter bots. We’ll first learn how to interface with the Twitter API using Twit.js, which allows us to post statuses, search tweets, and interact with followers. We’ll create several example bots that show the diversity of possibilities, using powerful tools such as RiTa.js, Google’s Cloud Vision API, and Tracery. We’ll learn how to tweet data from Google Spreadsheets, and to compose and tweet a song for an audio bot. We’ll also learn how to host our bot logic on Heroku and Docker. 

Twitter API Client for node

Supports both the REST and Streaming API.

twit

Create your own twitter bots

The Edge is one of the most exciting new topics in web development. At the front of the pack is Cloudflare, whose network boasts servers located in hundreds of cities all over the world instead of in regions like "East" or "West".

More servers in more locations mean a faster experience for your users.

But what about the data part of the equation?

If the application you're working on skews more toward "reading" rather than "writing" data, Cloudflare Workers KV is a perfect fit.

Once you create a KV namespace— the "KV" stands for "key-value"— Cloudflare keeps track of your application data. The most commonly accessed data will be propagated around the world to the various Edge servers, while less popular data will be stored in more central locations.

Kristian Freeman has produced courses that introduce how to build and deploy sites and JSON APIs powered by Cloudflare Workers.

Today, he will show you how Workers KV can be integrated into a Cloudflare Workers project through building a Todo App.

## The Project

![todo-app-screenshot](https://res.cloudinary.com/dg3gyk0gu/image/upload/v1639014074/course-resources/Build-data-driven-applications-on-the-edge-with-Workers-and-Workers-KV/CleanShot_2021-12-08_at_16.46.39_2x.png)

Before you groan about a bland Todo App, think about it this way: 

The Todo App is the perfect example of the CRUD operations. Cloudflare KV has its own nuances for reading and writing data through a concept called Binding that is easier to grok when you understand the larger example as a whole.

As the course progresses, Kristian shares examples of how you can store more complex key-values, along with data operations like editing and filtering.

One of the coolest features that Cloudflare KV allows for is user-specific data that is tied to the IP address the request originates from. In other words, you'll be able to share your deployed Todo app with anyone around the world, and they will only see their personal items.

## **Gain these Skills**

- Manipulating data using Cloudflare Workers KV
- Make data unique to each user without logging in
- Effectively store lightweight data in Cloudflare Workers KV
- Setting up a key-value store and connected projects
- Reading and writing to a serverless key-value store
- Storing complex values in a Cloudflare KV store

## **Prerequisites**

You'll need to have a basic understanding of what Cloudflare Workers is recommended but not necessary. This course is relatively beginner-friendly into the world of Workers KV that you can get by with no knowledge on Cloudflare Workers. 

But if you want to get the most out of this course, we would recommend taking [Introduction to Cloudflare Workers](https://egghead.io/courses/introduction-to-cloudflare-workers-5aa3) to get a better understanding of how Workers KV comes into play. 

## **What's Next? (You might also like these resources:)**

Follow along with Kristian Freeman as you build a localization engine that renders data based on the Edge location nearest to the application's user. → [Introduction to Cloudflare Workers](https://egghead.io/courses/introduction-to-cloudflare-workers-5aa3) 

Learn how to build your own serverless APIs so you can manage a highly available backend for your projects. No Learning DevOps or managing services necessary. →  [Build a Serverless API with Cloudflare Workers](https://egghead.io/courses/build-a-serverless-api-with-cloudflare-workers-d67ca551)

Using Cloudflare Pages and Cloudflare Workers, it's easy to create dynamic front ends that you can deploy in just seconds from GitHub. → [Deploy Dynamic Frontend Applications with Cloudflare Pages](https://egghead.io/courses/deploy-dynamic-frontend-applications-with-cloudflare-pages-6aaa00a1)

Build Data-Driven Applications on the Edge with Workers and Workers KV

Learn how to create dynamic front ends using both Cloudflare Pages and Cloudflare Workers. 

With Cloudflare Pages, you can deploy static sites in seconds from Github. In this course, you will learn how to get started by going through the setup process and integrating Contentful and Airtable. 

And we will the frontend dynamic by writing a serverless function with Cloudflare Workers. We set that up by using the Cloudflare Workers CLI, Wrangler, which makes it easy to connect to your Cloudflare account and create serverless functions. 

And then we connect them with our example project, where we need to send submitted form data from our front end over to Airtable. 

So check out this course and learn to use the Jamstack with Cloudflare today!

Deploy Dynamic Frontend Applications with Cloudflare Pages

We’ll get a real-time stream of tweets, and learn how to filter a stream by keyword, location, and user.

Use the Twitter Stream API with Twit.js

Nightmare is a high-level browser automation library.

The goal is to expose just a few simple methods, and have an API that feels synchronous for each block of scripting, rather than deeply nested callbacks. It's designed for automating tasks across sites that don't have APIs.

nightmare

Many pages you'll want to grab data from will require you to log in first. This video demonstrates how to log in and grab whatever data off the page you will need.

Scraping Data from Sites with Login Forms with Nightmare

Many websites have more than just simple static content. Dynamic content which is rendered by JavaScript requires browser to be able to scrape data. This video demonstrates how to use Nightmare (which is a wrapper around Electron) to launch a url and scrape dynamic data.

Scraping Dynamic JavaScript Websites with Nightmare

In this lesson, our bot will tweet hello to the Twitter world! We'll do this by using Twit's POST method to post our status to the statuses/update endpoint.

Post Statuses with Twit.js

In this lesson, we’ll create multiple functions to request, download, and save photos and data from NASA's API, and then have our bot upload these photos to Twitter and post them along with their descriptions. We'll also learn how to tweet videos using a video from NASA’s space archives.

Tweet Media Files with Twit.js

We’ll learn how your bot can get its list of followers, follow people, and look up friendships. We'll use Twit's GET method to get our followers at the followers/list endpoint, to get the users we follow at the friends/ids and friends/list endpoints, and to look up our friendships with the friendships/lookup endpoint. We'll also use Twit's POST method to follow someone at the friendships/create endpoint, and to send messages by posting to the direct_messages/new endpoint. 

Work with Users in Twit.js

We’ll learn the basics of interacting with tweets, including retweeting, deleting, favoriting, and replying to tweets. We'll get our home timeline by using Twit's GET method to access the `statuses/home_timeline` endpoint, including the count parameter, which lets us get back a certain number of tweets. We'll also pass it a callback. We'll learn how to cycle through the data (the tweets) we get back and see what information is included. We'll learn how to retweet statuses by posting to `statuses/retweet` and including the tweet id. We can unretweet by posting to `statuses/unretweet` with the same tweet id. We'll also learn how to like a tweet by posting to `favorites/create` with a tweet id, and unlike a tweet by posting to `favorites/destroy` with a tweet id. We'll also learn how to reply to a tweet by posting to `statuses/update`, with a status that includes the handle of the user we're replying to and `in_reply_to_status_id` parameter, which is the id of the tweet we're replying to. We'll learn how to delete a tweet by posting to `statuses/destroy` with the tweet's id. 

Interact with Tweets with Twit.js 

We’ll learn how to search tweets with the Twitter Search API, using Twit's GET method to get the search/tweets endpoint. We'll use the query and count parameters to get the search term(s) and number of tweets that we want. We'll learn how to get exact phrases, multiple words, one of several words, emoticons, hashtags, photos/videos, urls, and how to remove words from our results. We'll learn how to implement the safe filter and how to filter by media, website, or date. We'll learn how to get recent results, popular results, results by location, and results by language. 

The search API returns for relevance, and not completeness. If you want all the tweets from a search term, you should use the stream API (which we'll go over in the next lesson). 

Use the Twitter Search API with Twit.js

We’ll go over the basic Twitter bot setup, including how to start a project from scratch, install necessary packages, and store our private keys in environment variables. 

Set Up a Basic Twitter Bot with Twit.js

To deploy a Cloudflare Workers serverless function, you'll need to sign up for a free Cloudflare Workers account at [workers.cloudflare.com](http://workers.cloudflare.com/), and select your unique workers.dev subdomain to deploy your functions to.

Create a Cloudflare Workers Account

Using NPM, you can install Cloudflare Workers' CLI (command-line interface), Wrangler, by typing `npm install -g @cloudflare/wrangler` into your command line. Wrangler allows you to generate and publish Cloudflare Workers projects. We'll also use the `wrangler login` command which will give you a link to authorize Wrangler. This will set up an API token specifically for your Wrangler installation that will allow you to publish and read all the Workers and Workers KV information that you need.

Install and Configure the Cloudflare Workers CLI Wrangler

Wranglers generate command, `wrangler generate`, allows you to create new Cloudflare Workers projects. Using the URL parameter, you can also generate new projects based on Cloudflare's starter templates, or open-source projects created by other developers, but for now, you will just use the default. To do this you'll only need to give it a name, for example, `wrangler generate my-worker`,  this will create a new project called my-worker. 

Generate New Cloudflare Workers Projects with Wrangler's generate Command

We'll take a look at the default code in `index.js` in a Workers project. We'll take a second to understand how it works before building the rest of our project. 

First will see what our Worker looks like in an actual browser by running `wrangler preview`. In a terminal, and we'll check out how Serverless functions in Cloudflare Workers listen for HTTPS requests via the "fetch" event and respond to them by creating and returning a new Response.

Write Your First Cloudflare Workers Serverless Function

Using `wrangler whoami` in a terminal we'll grab our account id so we can paste our id into the double quotes of `account_id = ""`  inside of `wrangler.toml` file. Using the `wrangler dev` command we can look at our Workers serverless function and test it before deploying it. Once we're ready to deploy, we can use the `wrangler publish` command to deploy it to our unique workers.dev subdomain.

Preview and Publish Your Cloudflare Workers Project

By updating the response body and setting a corresponding Content-Type header, we can render HTML pages from directly inside of our Cloudflare Workers function.

We will update our Cloudflare Worker to return a call to a template function that we define. That template function returns a string of HTML that we will render in the browser.

Render HTML Pages with Cloudflare Workers

Up until now, we've been rendering plain responses, not based on any information coming in with a request. The request.cf object sent with every incoming HTTP request to a Workers function has a ton of interesting information, including the regionality of the request, and which Cloudflare data center is responding to the request. Using this information, we'll render a dynamic HTML page based on where the user visiting our site is in the world.