Learn Jekyll, Mongodb and Nightmare

MongoDB has a quite powerful aggregation framework that allows you to write data processing pipelines. Documents in the MongoDB collection go through a series of stages or a pipeline that transforms them into an aggregated result. In this short video series, you will learn how to write aggregations in MongoDB.

mongodb

MongoDB Aggregation Framework

Jekyll is a blog-aware static site generator. You can write text in markdown or HTML and Jekyll uses layouts to create a static website. You can tweak how you want the site URLs to look, what data gets displayed on the site, and more.

After watching the collection you'll be able to:

* Install Jeykll and run it locally
* Create a new blog post
* Install plugins to add new features to your blog

Jekyll is a parsing engine bundled as a ruby gem used to build static websites from dynamic components such as templates, partials, liquid code, markdown, etc. Jekyll is known as "a simple, blog aware, static site generator".

jekyll

Create A Blog With Jekyll

Aggregation in MongoDB is really powerful and allows you to create solid pipelines to extract and transform data. This is an introductory video that aims to demonstrate the basics of aggregation — we will see how to create some basic pipeline to extract data using `$match` and manipulate the output using `$project` stages.

The aggregation stages/operators covered in this video are:

* [`$match`](https://docs.mongodb.com/manual/reference/operator/aggregation/match/index.html)
* [`$project`](https://docs.mongodb.com/manual/reference/operator/aggregation/project/index.html)
* [`$concat`](https://docs.mongodb.com/manual/reference/operator/aggregation/concat/)

You can see the list of all the [available operators in the MongoDB docs](https://docs.mongodb.com/manual/reference/operator/aggregation/)


Find and Manipulate Data using Aggregation in MongoDB

In our last lesson, we learnt the basics of MongoDB aggregation and saw how we can [find and manipulate data](https://egghead.io/lessons/mongodb-find-and-manipulate-data-using-aggregation-in-mongodb). In this continuation, we will see how we can sort, skip and limit records during our aggregation stages.

The aggregation stages/operators covered in this video are:

* [`$sort`](https://docs.mongodb.com/manual/reference/operator/aggregation/sort/index.html)
* [`$skip`](https://docs.mongodb.com/manual/reference/operator/aggregation/skip/index.html)
* [`$limit`](https://docs.mongodb.com/manual/reference/operator/aggregation/limit/index.html)

You can see the list of all the [available operators in the MongoDB docs.](https://docs.mongodb.com/manual/reference/operator/aggregation/)

Sort, Skip and Limit Records in MongoDB Aggregation

Mongoose provides a straight-forward, schema-based solution to model your application data. It includes built-in type casting, validation, query building, business logic hooks and more, out of the box.

mongoose

One of the great things about NoSQL databases is that you do not have to decide on your data schema while developing. As you get closer to production though, you'll often want to more clearly define the shape of the data you are working with.

Enforce Schemas in MongoDB using Mongoose Models

Unlike the relational databases, MongoDB doesn't allow you to join collections using the simple `find` queries, but you can write the join queries using the aggregation framework. In this lesson, we learn how to join two collections to get the data in the format we want.

The aggregation operators that we cover in this video are:

- [`$lookup`](https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/index.html)
- [`$project`](https://docs.mongodb.com/manual/reference/operator/aggregation/project/index.html)
- [`$arrayElemAt`](https://docs.mongodb.com/manual/reference/operator/aggregation/arrayElemAt/index.html)

You can see the list of all the [available operators in the MongoDB docs.](https://docs.mongodb.com/manual/reference/operator/aggregation/)

Write Joins in MongoDB

[In our last lesson](https://egghead.io/lessons/mongodb-group-by-and-sum-in-mongodb?pl=mongodb-aggregation-framework-3e5d), we saw how we can use `$group` operator to simulate `GROUP BY` operations that you might be aware from SQL. A minor gimmick with `$group` is that it only works with the scalar field values and not arrays. There might be times when you would want to group by some array fields; in this lesson, we learn how to do that using `$unwind` operator

Here is the list of operators that we cover in this lesson:

- [$unwind](https://docs.mongodb.com/manual/reference/operator/aggregation/unwind/index.html)
- [$group](https://docs.mongodb.com/manual/reference/operator/aggregation/group/index.html)
- [$sort](https://docs.mongodb.com/manual/reference/operator/aggregation/sort/index.html)
- [$limit](https://docs.mongodb.com/manual/reference/operator/aggregation/limit/index.html)

You can see the list of all the [available operators in the MongoDB docs](https://docs.mongodb.com/manual/reference/operator/aggregation/).


Perform Group Operations on Array Fields in MongoDB

In this lesson, we learn how to group documents in a collection using `$group` operator similar to `GROUP BY` in SQL. We also look at `$sum` operator and see how we can use it to count the number of documents as well as get sum of some field in our grouped results.

Here is the list of operators that we cover in this lesson:

* [`$group`](https://docs.mongodb.com/manual/reference/operator/aggregation/group/index.html)
* [`$sum`](https://docs.mongodb.com/manual/reference/operator/aggregation/sum/index.html)
* [`$sort`](https://docs.mongodb.com/manual/reference/operator/aggregation/sort/index.html)

You can see the list of all the [available operators in the MongoDB docs](https://docs.mongodb.com/manual/reference/operator/aggregation/).


Group By and Sum in MongoDB

Jekyll is a blog-aware, static site generator written in Ruby. 

Requirements:
* Ruby version 2.5.0 or above
* RubyGems

Install the Jekyll Ruby gem on macOS quickly with just a few terminal commands and have a blog-aware site running locally.

Install Jekyll and Create a New Project

Blogging is built into Jekyll. You write blog posts as markdown or HTML files and Jekyll turns them blog posts. They follow a specific naming convention to be picked up and used by Jekyll

All blog post files require front matter. Front matter is a snippet of YAML at the top of a file. Front matter is used to set your layouts and meta data.

Create New Blog Post with Jekyll

Jekyll has a plugin system that allows you to add custom features to your site. There are many plugins available or you can even write your own. You can view a list of plugins at [Planet Jekyll](https://github.com/planetjekyll/awesome-jekyll-plugins)

Liquid is a templating language written in Ruby to output to the browser 

[Liquid Documentation](https://shopify.github.io/liquid/)

Install a Jekyll Plugin

Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications.

express

Mongoose allows you to easily select resources by ID from your MongoDB. This is an important aspect to creating an API.

Select by ID with Mongoose and Express

Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

node

Learn how to import data into your MongoDB and then use Express to serve a simple Node.js API.

First API with Node.js, Express and MongoDB

Nightmare is a high-level browser automation library.

The goal is to expose just a few simple methods, and have an API that feels synchronous for each block of scripting, rather than deeply nested callbacks. It's designed for automating tasks across sites that don't have APIs.

nightmare

Many pages you'll want to grab data from will require you to log in first. This video demonstrates how to log in and grab whatever data off the page you will need.

Scraping Data from Sites with Login Forms with Nightmare

Many websites have more than just simple static content. Dynamic content which is rendered by JavaScript requires browser to be able to scrape data. This video demonstrates how to use Nightmare (which is a wrapper around Electron) to launch a url and scrape dynamic data.

Scraping Dynamic JavaScript Websites with Nightmare

[Jekyll](https://jekyllrb.com/) is a static site generator built in [Ruby](https://www.ruby-lang.org/en/) and available to generate and host websites on [GitHub Pages](https://pages.github.com/). In this course, you'll install the Jekyll gem, use the command line tool to generate a new project, inspect the scaffolded project, and finally, serve the starter website.

Topics

Instructors

Content Type