Learn Chai, Jq and Nightmare

jq is a lightweight and flexible command-line JSON processor.

jq can help you quickly filter massive amounts of JSON data, but how can you turn it into a CSV, so you can process it quickly with a database or spreadsheet application?

This lesson shows how to select the data you want for your CSV with jq, and uses the helper utility linked below to output as CSV.

https://github.com/joelpurra/jq-hopkok/tree/master/src/tabular

Convert JSON to CSV with jq

Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.

chai

[Chai](https://npmjs.com/package/chai) assertions work great with the [Mocha](https://npmjs.com/package/mocha) testing framework. In this lesson we'll walk through out to install Chai and [karma-chai](https://npmjs.com/package/karma-chai) so you can use the `expect` assertions in your tests.

Use Chai assertions for tests in a Karma project

Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.

mocha

Every dependable library needs to have tests to ensure that you don't push accidental bugs. In this lesson, see how you can set up mocha and chai to start testing your library.

Setting up Unit Testing with Mocha and Chai

Now that you've got everything set up with mocha and chai, it's time to actually write tests. In this lesson you'll learn how to use `describe` and `it` to outline your tests and make sure you don't break your library in the future.

Unit Testing with Mocha and Chai

Nightmare is a high-level browser automation library.

The goal is to expose just a few simple methods, and have an API that feels synchronous for each block of scripting, rather than deeply nested callbacks. It's designed for automating tasks across sites that don't have APIs.

nightmare

Many pages you'll want to grab data from will require you to log in first. This video demonstrates how to log in and grab whatever data off the page you will need.

Scraping Data from Sites with Login Forms with Nightmare

In this lesson, we will use Chai's request method to test our Node application's API responses.
By the end of this lesson, you will know how to:
- install the prerequisites to use mocha and chai in your application
- test for HTTP status response codes
- test for a string of text on a page
- test for a json response and validate the properties of the object
- write tests that not only verify the response of your application, but the behavior as well


Test Node RESTful API with Mocha and Chai

jq is a lightweight and flexible command-line JSON processor. It is extremely useful for processing data from APIs to better understand the data your code will be consuming, before you write the code. In this lesson I introduce you to jq and show you the basic features it provides using the Star Wars API (http://swapi.co) as our data source. Topics covered include filtering, piping, object construction and data manipulation.

Using jq to parse the Star Wars API

Many websites have more than just simple static content. Dynamic content which is rendered by JavaScript requires browser to be able to scrape data. This video demonstrates how to use Nightmare (which is a wrapper around Electron) to launch a url and scrape dynamic data.

Scraping Dynamic JavaScript Websites with Nightmare

Writing great ES6 style Promises for Node.js is only half the battle. Your great modules must include tests as well to ensure future iterations don't break them. In this lesson, I show you a simple ES6 Promise in Node.js, then walk you through creating tests in Mocha using chai and chai-as-promised to test both resolve and reject methods.

Testing ES6 Promises in Node.js using Mocha and Chai

In this lesson, you will learn how to verify your API server is returning HTTP 400 responses when clients submit incorrect data. Returning HTTP 400 ensures that your clients are notified of the incorrect usage. Testing for them ensures your API returns errors instead of incorrect responses when supplied with incorrect data.

Test for HTTP 400 responses with Mocha and Chai

Bash, unfortunately, doesn’t ship with a command that can work with JSON natively. In this lesson, we’ll learn how to read and do basic queries on JSON with `jq`, an installable command line tool that makes working with JSON in bash really easy. We'll pipe the JSON output of the Github API to jq to extract values. 

Note: jq has to be installed. On macOS, the easiest way to do this is to run `brew install jq`.  To view install instructions for all platforms, see https://stedolan.github.io/jq/download/. 

Read and Use JSON in Bash with jq

In this lesson, we'll apply our `jq` skills and write a script that gets a list of dependencies from package.json and greps for usages of every package. If a package is unused, it's marked for removal. 

Topics

Instructors

Content Type