Learn Jq, Linux and Nightmare

jq is a lightweight and flexible command-line JSON processor.

jq can help you quickly filter massive amounts of JSON data, but how can you turn it into a CSV, so you can process it quickly with a database or spreadsheet application?

This lesson shows how to select the data you want for your CSV with jq, and uses the helper utility linked below to output as CSV.

https://github.com/joelpurra/jq-hopkok/tree/master/src/tabular

Convert JSON to CSV with jq

Nightmare is a high-level browser automation library.

The goal is to expose just a few simple methods, and have an API that feels synchronous for each block of scripting, rather than deeply nested callbacks. It's designed for automating tasks across sites that don't have APIs.

nightmare

Many pages you'll want to grab data from will require you to log in first. This video demonstrates how to log in and grab whatever data off the page you will need.

Scraping Data from Sites with Login Forms with Nightmare

jq is a lightweight and flexible command-line JSON processor. It is extremely useful for processing data from APIs to better understand the data your code will be consuming, before you write the code. In this lesson I introduce you to jq and show you the basic features it provides using the Star Wars API (http://swapi.co) as our data source. Topics covered include filtering, piping, object construction and data manipulation.

Using jq to parse the Star Wars API

Many websites have more than just simple static content. Dynamic content which is rendered by JavaScript requires browser to be able to scrape data. This video demonstrates how to use Nightmare (which is a wrapper around Electron) to launch a url and scrape dynamic data.

Scraping Dynamic JavaScript Websites with Nightmare

Installing the AWS CLI on your local workstation may be one of the biggest boosts to productivity you can make if you use AWS often. Almost all AWS features are available via the CLI allowing you to script, schedule and automate many AWS related tasks.
In this lesson, I’ll show you how to setup and verify the AWS CLI.

Installing the AWS CLI on OS X and Linux

Vim is a highly configurable text editor built to make creating and changing any kind of text very efficient. It is included as "vi" with most UNIX systems and with Apple OS X.

Every programmer starting to work in a *nix environment eventually finds themselves in VIM, the default text editor. Often by mistake! Because VIM lacks the typical polished GUI of normal programs, it can be hard to discover the correct sequence of commands to even exit the program. Once you learn how to exit VIM, then the next task is to also learn how to make VIM do what you want -while- exiting!

In this lesson we cover 7 different ways of exiting VIM, whether you are in EX mode or Command (also known as Escape) mode, whether you are editing a single file or several at once.

Exit Vim

Bash, unfortunately, doesn’t ship with a command that can work with JSON natively. In this lesson, we’ll learn how to read and do basic queries on JSON with `jq`, an installable command line tool that makes working with JSON in bash really easy. We'll pipe the JSON output of the Github API to jq to extract values. 

Note: jq has to be installed. On macOS, the easiest way to do this is to run `brew install jq`.  To view install instructions for all platforms, see https://stedolan.github.io/jq/download/. 

Read and Use JSON in Bash with jq

Flutter allows you to build beautiful native apps on iOS and Android from a single codebase.

flutter

We will setup our first command-line Dart application. We will use the apt-get package manager on Ubuntu to install the SDK and write our first program. This will prepare you to follow along in the next lessons.

Get the gist of commands → https://gist.github.com/creativebracket/b44ee73739f249be0d5f4b8676e8eca5

Set up your first Dart project on Linux

In this lesson, we'll apply our `jq` skills and write a script that gets a list of dependencies from package.json and greps for usages of every package. If a package is unused, it's marked for removal. 

Topics

Instructors

Content Type