Learn Jq, Mongodb and Redwoodjs

MongoDB has a quite powerful aggregation framework that allows you to write data processing pipelines. Documents in the MongoDB collection go through a series of stages or a pipeline that transforms them into an aggregated result. In this short video series, you will learn how to write aggregations in MongoDB.

mongodb

MongoDB Aggregation Framework

RedwoodJS takes the JAMstack philosophy to the next level. It brings together all the tools modern web development has to offer:

- React
- GraphQL
- Prisma

RedwoodJS abstracts database management away from you so that you can just build your schema and it will do all the plumbing. This collection will go over how to create a RedwoodJS project and the scripts that will help make your developer experience a joy.


Bringing full-stack to the JAMstack

Do you love the JAMstack philosophy but need a database-backed web app? Want great developer experience and easy scaling? Redwood is here! Built on React, GraphQL, and Prisma, Redwood works with the components and development workflow you love, but with simple conventions and helpers to make your experience even better.

redwoodjs

Introduction to RedwoodJS: full-stack framework for JAMstack

jq is a lightweight and flexible command-line JSON processor.

jq can help you quickly filter massive amounts of JSON data, but how can you turn it into a CSV, so you can process it quickly with a database or spreadsheet application?

This lesson shows how to select the data you want for your CSV with jq, and uses the helper utility linked below to output as CSV.

https://github.com/joelpurra/jq-hopkok/tree/master/src/tabular

Convert JSON to CSV with jq

Aggregation in MongoDB is really powerful and allows you to create solid pipelines to extract and transform data. This is an introductory video that aims to demonstrate the basics of aggregation — we will see how to create some basic pipeline to extract data using `$match` and manipulate the output using `$project` stages.

The aggregation stages/operators covered in this video are:

* [`$match`](https://docs.mongodb.com/manual/reference/operator/aggregation/match/index.html)
* [`$project`](https://docs.mongodb.com/manual/reference/operator/aggregation/project/index.html)
* [`$concat`](https://docs.mongodb.com/manual/reference/operator/aggregation/concat/)

You can see the list of all the [available operators in the MongoDB docs](https://docs.mongodb.com/manual/reference/operator/aggregation/)


Find and Manipulate Data using Aggregation in MongoDB

React is one of the web’s most popular frameworks for building JavaScript applications. 

If you know what you’re doing, React can drastically simplify how you build, use, and maintain code.

Whether you’re a React newbie or you’re ready for advanced techniques, you can level-up with egghead. Explore our in-depth courses, lessons, and community resources to build more powerful applications and crack open your career.

react

We use `yarn redwood generate page` (`yarn rw` for short) in this lesson to generate an `IndexPage` component. 

The first argument to `yarn rw generate page` is the name of the page you are generating. The second argument is the path you want the page to show up on. So `yarn rw generate page index /` will generate an `IndexPage` component and insert a route entry in our `web/src/Routes.js` file.

We learn that redwood import `React` for you so you don't have to import it manually when you are creating a component.

Generate a Page Using the Redwood CLI

In this lesson, we are going over how to generate a `RedwoodJS` project.

We can create a `RedwoodJS` project with `yarn create redwood-app ./my-redwood-project`.

`yarn` will generate the `RedwoodJS` app in the file folder you specified without installing any globals.

To run the `RedwoodJS` app, we use `yarn redwood dev`. This runs the Redwood client as well as the `GraphQL` backend.

Generate a Redwood Project

RedwoodJS provides a `Link` component that we can use to navigate around our app. We can provide a `to` prop with the path that we want to navigate to.

RedwoodJS takes this a step further and provides a `routes` object from `@redwoodjs/router`. This object is aware of all the routes defined in `web/Routes.js`. This allows us to use the methods on this object rather than referencing the our routes with a string literal. We can then change the route path without having to change all of the references to that route all over our app.

Use RedwoodJS Link and routes Object to Navigate

RedwoodJS gives us a cli command to generate layouts.

`yarn redwood generate layout main`

This will generate a `MainLayout` component in `src/layouts/MainLayout/`. We can use this layout component to define any headers or footers we want to share across multiple pages. 

Extract a RedwoodJS Layout Component

In our last lesson, we learnt the basics of MongoDB aggregation and saw how we can [find and manipulate data](https://egghead.io/lessons/mongodb-find-and-manipulate-data-using-aggregation-in-mongodb). In this continuation, we will see how we can sort, skip and limit records during our aggregation stages.

The aggregation stages/operators covered in this video are:

* [`$sort`](https://docs.mongodb.com/manual/reference/operator/aggregation/sort/index.html)
* [`$skip`](https://docs.mongodb.com/manual/reference/operator/aggregation/skip/index.html)
* [`$limit`](https://docs.mongodb.com/manual/reference/operator/aggregation/limit/index.html)

You can see the list of all the [available operators in the MongoDB docs.](https://docs.mongodb.com/manual/reference/operator/aggregation/)

Sort, Skip and Limit Records in MongoDB Aggregation

Mongoose provides a straight-forward, schema-based solution to model your application data. It includes built-in type casting, validation, query building, business logic hooks and more, out of the box.

mongoose

One of the great things about NoSQL databases is that you do not have to decide on your data schema while developing. As you get closer to production though, you'll often want to more clearly define the shape of the data you are working with.

Enforce Schemas in MongoDB using Mongoose Models

Unlike the relational databases, MongoDB doesn't allow you to join collections using the simple `find` queries, but you can write the join queries using the aggregation framework. In this lesson, we learn how to join two collections to get the data in the format we want.

The aggregation operators that we cover in this video are:

- [`$lookup`](https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/index.html)
- [`$project`](https://docs.mongodb.com/manual/reference/operator/aggregation/project/index.html)
- [`$arrayElemAt`](https://docs.mongodb.com/manual/reference/operator/aggregation/arrayElemAt/index.html)

You can see the list of all the [available operators in the MongoDB docs.](https://docs.mongodb.com/manual/reference/operator/aggregation/)

Write Joins in MongoDB

[In our last lesson](https://egghead.io/lessons/mongodb-group-by-and-sum-in-mongodb?pl=mongodb-aggregation-framework-3e5d), we saw how we can use `$group` operator to simulate `GROUP BY` operations that you might be aware from SQL. A minor gimmick with `$group` is that it only works with the scalar field values and not arrays. There might be times when you would want to group by some array fields; in this lesson, we learn how to do that using `$unwind` operator

Here is the list of operators that we cover in this lesson:

- [$unwind](https://docs.mongodb.com/manual/reference/operator/aggregation/unwind/index.html)
- [$group](https://docs.mongodb.com/manual/reference/operator/aggregation/group/index.html)
- [$sort](https://docs.mongodb.com/manual/reference/operator/aggregation/sort/index.html)
- [$limit](https://docs.mongodb.com/manual/reference/operator/aggregation/limit/index.html)

You can see the list of all the [available operators in the MongoDB docs](https://docs.mongodb.com/manual/reference/operator/aggregation/).


Perform Group Operations on Array Fields in MongoDB

In this lesson, we learn how to group documents in a collection using `$group` operator similar to `GROUP BY` in SQL. We also look at `$sum` operator and see how we can use it to count the number of documents as well as get sum of some field in our grouped results.

Here is the list of operators that we cover in this lesson:

* [`$group`](https://docs.mongodb.com/manual/reference/operator/aggregation/group/index.html)
* [`$sum`](https://docs.mongodb.com/manual/reference/operator/aggregation/sum/index.html)
* [`$sort`](https://docs.mongodb.com/manual/reference/operator/aggregation/sort/index.html)

You can see the list of all the [available operators in the MongoDB docs](https://docs.mongodb.com/manual/reference/operator/aggregation/).


Group By and Sum in MongoDB

Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications.

express

Mongoose allows you to easily select resources by ID from your MongoDB. This is an important aspect to creating an API.

Select by ID with Mongoose and Express

Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

node

Learn how to import data into your MongoDB and then use Express to serve a simple Node.js API.

First API with Node.js, Express and MongoDB

jq is a lightweight and flexible command-line JSON processor. It is extremely useful for processing data from APIs to better understand the data your code will be consuming, before you write the code. In this lesson I introduce you to jq and show you the basic features it provides using the Star Wars API (http://swapi.co) as our data source. Topics covered include filtering, piping, object construction and data manipulation.

Using jq to parse the Star Wars API

Bash, unfortunately, doesn’t ship with a command that can work with JSON natively. In this lesson, we’ll learn how to read and do basic queries on JSON with `jq`, an installable command line tool that makes working with JSON in bash really easy. We'll pipe the JSON output of the Github API to jq to extract values. 

Note: jq has to be installed. On macOS, the easiest way to do this is to run `brew install jq`.  To view install instructions for all platforms, see https://stedolan.github.io/jq/download/. 

Read and Use JSON in Bash with jq

In this lesson, we'll apply our `jq` skills and write a script that gets a list of dependencies from package.json and greps for usages of every package. If a package is unused, it's marked for removal. 

Topics

Instructors

Content Type