Learn Bash, Nightmare and Twit

🚀 In this collection, I will be going over some command line tools I use on a daily basis that make my life a lot easier.

[First up is fzf](https://github.com/junegunn/fzf).

> fzf is a general-purpose command-line fuzzy finder.

Bash is a Unix shell and command language written by Brian Fox for the GNU Project as a free software replacement for the Bourne shell. Bash is a command processor that typically runs in a text window, where the user types commands that cause actions. Bash can also read and execute commands from a file, called a shell script.

bash

Upgrade Your Terminal and Make It a Joy to Work with

Extracting meaningful data from strings is a crucial skill when working with bash, and bash gives us powerful tools to do so: `head`, `tail`, `cut`, `sed`, and `awk`. But these tools can be hard to understand without spending time to learn them, so many developers fall into a rut of blindly copying and pasting commands. No more! With this mini-course, you'll understand the basic ins and outs of these commands in under 15 minutes. 


Manipulating strings in bash

SSH provides a secure way for your to [connect to and authenticate with GitHub's servers](https://help.github.com/en/github/authenticating-to-github/about-ssh). While HTTPS connections use usernames and passwords as credentials, SSH instead uses public and private key pairs. These can be generated on your system using a CLI like [`ssh-keygen`](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) and then associated with your GitHub account using the public key.

In some cases, you may need to connect to more than one GitHub account from the same system. For example, you may have [separate accounts for personal and professional projects](https://ryanharris.dev/managing-ssh-keys-for-github/), which can be difficult to manage manually. Using `ssh-agent` and a SSH configuration file, you can alleviate this pain point by automatically determining which key to use based on your GitHub repository's `git` URL.

Secure GitHub connections with SSH

“Setting up remote server authentication” will never be on your job description. It’s not one of those desirable responsibilities that convinces talented people to move across the country and pursue the job of their dreams.

But at that dream job, you might be asked to set up an SSH key. You’d better be ready.

In this course, we will learn the fundamentals of creating SSH keys and configuring them for connectivity to a remote server. We’ll figure out how to:

- tunnel SSH connections for easier local debugging
- set up configuration files to simplify the aspect of connecting to remote servers
- use security features like blocking incoming connections

Being prepared is key. Learn the skills now!

SSH for Remote Server Authentication

Bash makes automation easy. Just a few lines of code can take care of repetitive tasks for you and your teammates. 

In this course, we’ll dive deep into the automation potential of bash. You’ll learn to leverage common hooks that open up interesting automation possibilities, like git hooks, adding functions and aliases in your bash_profile, and running scheduled jobs in launchd.

For example, we’ll write a git hook that checks package.json for changes, and if there are changes, the hook runs npm install for you automatically. 

Along the way, you’ll also learn:

- more advanced bash commands like exec
- syntax such as redirecting stdout and stderr to the same destination
- timesaving tricks like brace expansions

Automation is all about saving you time — and at only 52 minutes, this course is well worth yours. Enjoy!

Advanced Bash Automation for Web Developers



Humans were never meant to repeat joyless tasks in a precise manner, over and over and over. Computers are meant precisely for these types of tasks.

Bash is a ubiquitous technology in web development that gives you the immense power of automating repetitive tasks you find yourself performing throughout the day.

The topics we cover will give you confidence in creating build scripts that run on continuous integration servers or quickly testing and debugging an API with `curl`. We’ll go over:  - Interacting with the file system - Creating, moving, copying, and deleting files and folders - Finding files and text patterns in files - Creating and running scripts - Language syntax like variables and conditionals.

Automate Daily Development Tasks with Bash

In this course we’ll learn how to create Twitter bots. We’ll first learn how to interface with the Twitter API using Twit.js, which allows us to post statuses, search tweets, and interact with followers. We’ll create several example bots that show the diversity of possibilities, using powerful tools such as RiTa.js, Google’s Cloud Vision API, and Tracery. We’ll learn how to tweet data from Google Spreadsheets, and to compose and tweet a song for an audio bot. We’ll also learn how to host our bot logic on Heroku and Docker. 

Twitter API Client for node

Supports both the REST and Streaming API.

twit

Create your own twitter bots

We’ll get a real-time stream of tweets, and learn how to filter a stream by keyword, location, and user.

Use the Twitter Stream API with Twit.js

Nightmare is a high-level browser automation library.

The goal is to expose just a few simple methods, and have an API that feels synchronous for each block of scripting, rather than deeply nested callbacks. It's designed for automating tasks across sites that don't have APIs.

nightmare

Many pages you'll want to grab data from will require you to log in first. This video demonstrates how to log in and grab whatever data off the page you will need.

Scraping Data from Sites with Login Forms with Nightmare

Many websites have more than just simple static content. Dynamic content which is rendered by JavaScript requires browser to be able to scrape data. This video demonstrates how to use Nightmare (which is a wrapper around Electron) to launch a url and scrape dynamic data.

Scraping Dynamic JavaScript Websites with Nightmare

In this lesson, our bot will tweet hello to the Twitter world! We'll do this by using Twit's POST method to post our status to the statuses/update endpoint.

Post Statuses with Twit.js

In this lesson, we’ll create multiple functions to request, download, and save photos and data from NASA's API, and then have our bot upload these photos to Twitter and post them along with their descriptions. We'll also learn how to tweet videos using a video from NASA’s space archives.

Tweet Media Files with Twit.js

We’ll learn how your bot can get its list of followers, follow people, and look up friendships. We'll use Twit's GET method to get our followers at the followers/list endpoint, to get the users we follow at the friends/ids and friends/list endpoints, and to look up our friendships with the friendships/lookup endpoint. We'll also use Twit's POST method to follow someone at the friendships/create endpoint, and to send messages by posting to the direct_messages/new endpoint. 

Work with Users in Twit.js

We’ll learn the basics of interacting with tweets, including retweeting, deleting, favoriting, and replying to tweets. We'll get our home timeline by using Twit's GET method to access the `statuses/home_timeline` endpoint, including the count parameter, which lets us get back a certain number of tweets. We'll also pass it a callback. We'll learn how to cycle through the data (the tweets) we get back and see what information is included. We'll learn how to retweet statuses by posting to `statuses/retweet` and including the tweet id. We can unretweet by posting to `statuses/unretweet` with the same tweet id. We'll also learn how to like a tweet by posting to `favorites/create` with a tweet id, and unlike a tweet by posting to `favorites/destroy` with a tweet id. We'll also learn how to reply to a tweet by posting to `statuses/update`, with a status that includes the handle of the user we're replying to and `in_reply_to_status_id` parameter, which is the id of the tweet we're replying to. We'll learn how to delete a tweet by posting to `statuses/destroy` with the tweet's id. 

Interact with Tweets with Twit.js 

We’ll learn how to search tweets with the Twitter Search API, using Twit's GET method to get the search/tweets endpoint. We'll use the query and count parameters to get the search term(s) and number of tweets that we want. We'll learn how to get exact phrases, multiple words, one of several words, emoticons, hashtags, photos/videos, urls, and how to remove words from our results. We'll learn how to implement the safe filter and how to filter by media, website, or date. We'll learn how to get recent results, popular results, results by location, and results by language. 

The search API returns for relevance, and not completeness. If you want all the tweets from a search term, you should use the stream API (which we'll go over in the next lesson). 

Use the Twitter Search API with Twit.js

We’ll go over the basic Twitter bot setup, including how to start a project from scratch, install necessary packages, and store our private keys in environment variables. 

Set Up a Basic Twitter Bot with Twit.js

In this lesson, we’ll learn how to change the working directory using `cd` and list a directory's contents using `ls`. We'll see how `ls`'s `-l` and `-a` flags work. 

Navigate the Filesystem in Bash

Sometimes when working at the command line, it can be handy to view a file’s contents right in the terminal, or open a file with a certain application. We’ll learn how to view files in the terminal using `cat` and `less` and we’ll learn how to open them with `open`.

View Files and Folders in Bash

One of the most useful things you can automate with bash is creating and deleting files and folders. We’ll learn how to create a file with `touch` and append content to it with `echo`. Then we’ll create a directory with `mkdir`. Using `rm`, we’ll then remove files and folders. 

Create and Delete Files and Folders in Bash

In this lesson we’ll learn how to move and rename files (`mv`) and copy (`cp`) them. 

Move and Copy Files and Folders with Bash

`find` is a powerful tool that can not only find files but it can run a command on each matching file too. In this lesson, we’ll learn how to find all the images that match a pattern and run an image optimization tool on them. 