Learn Jekyll, Jq and Nightmare

Jekyll is a blog-aware static site generator. You can write text in markdown or HTML and Jekyll uses layouts to create a static website. You can tweak how you want the site URLs to look, what data gets displayed on the site, and more.

After watching the collection you'll be able to:

* Install Jeykll and run it locally
* Create a new blog post
* Install plugins to add new features to your blog

Jekyll is a parsing engine bundled as a ruby gem used to build static websites from dynamic components such as templates, partials, liquid code, markdown, etc. Jekyll is known as "a simple, blog aware, static site generator".

jekyll

Create A Blog With Jekyll

jq is a lightweight and flexible command-line JSON processor.

jq can help you quickly filter massive amounts of JSON data, but how can you turn it into a CSV, so you can process it quickly with a database or spreadsheet application?

This lesson shows how to select the data you want for your CSV with jq, and uses the helper utility linked below to output as CSV.

https://github.com/joelpurra/jq-hopkok/tree/master/src/tabular

Convert JSON to CSV with jq

Jekyll is a blog-aware, static site generator written in Ruby. 

Requirements:
* Ruby version 2.5.0 or above
* RubyGems

Install the Jekyll Ruby gem on macOS quickly with just a few terminal commands and have a blog-aware site running locally.

Install Jekyll and Create a New Project

Blogging is built into Jekyll. You write blog posts as markdown or HTML files and Jekyll turns them blog posts. They follow a specific naming convention to be picked up and used by Jekyll

All blog post files require front matter. Front matter is a snippet of YAML at the top of a file. Front matter is used to set your layouts and meta data.

Create New Blog Post with Jekyll

Jekyll has a plugin system that allows you to add custom features to your site. There are many plugins available or you can even write your own. You can view a list of plugins at [Planet Jekyll](https://github.com/planetjekyll/awesome-jekyll-plugins)

Liquid is a templating language written in Ruby to output to the browser 

[Liquid Documentation](https://shopify.github.io/liquid/)

Install a Jekyll Plugin

Nightmare is a high-level browser automation library.

The goal is to expose just a few simple methods, and have an API that feels synchronous for each block of scripting, rather than deeply nested callbacks. It's designed for automating tasks across sites that don't have APIs.

nightmare

Many pages you'll want to grab data from will require you to log in first. This video demonstrates how to log in and grab whatever data off the page you will need.

Scraping Data from Sites with Login Forms with Nightmare

jq is a lightweight and flexible command-line JSON processor. It is extremely useful for processing data from APIs to better understand the data your code will be consuming, before you write the code. In this lesson I introduce you to jq and show you the basic features it provides using the Star Wars API (http://swapi.co) as our data source. Topics covered include filtering, piping, object construction and data manipulation.

Using jq to parse the Star Wars API

Many websites have more than just simple static content. Dynamic content which is rendered by JavaScript requires browser to be able to scrape data. This video demonstrates how to use Nightmare (which is a wrapper around Electron) to launch a url and scrape dynamic data.

Scraping Dynamic JavaScript Websites with Nightmare

[Jekyll](https://jekyllrb.com/) is a static site generator built in [Ruby](https://www.ruby-lang.org/en/) and available to generate and host websites on [GitHub Pages](https://pages.github.com/). In this course, you'll install the Jekyll gem, use the command line tool to generate a new project, inspect the scaffolded project, and finally, serve the starter website.

Install Jekyll and generate initial, scaffolded website

Bash, unfortunately, doesn’t ship with a command that can work with JSON natively. In this lesson, we’ll learn how to read and do basic queries on JSON with `jq`, an installable command line tool that makes working with JSON in bash really easy. We'll pipe the JSON output of the Github API to jq to extract values. 

Note: jq has to be installed. On macOS, the easiest way to do this is to run `brew install jq`.  To view install instructions for all platforms, see https://stedolan.github.io/jq/download/. 

Read and Use JSON in Bash with jq

In this lesson, we'll apply our `jq` skills and write a script that gets a list of dependencies from package.json and greps for usages of every package. If a package is unused, it's marked for removal. 

Topics

Instructors

Content Type