Learn Elasticsearch, Lodash and Twit

In this course we’ll learn how to create Twitter bots. We’ll first learn how to interface with the Twitter API using Twit.js, which allows us to post statuses, search tweets, and interact with followers. We’ll create several example bots that show the diversity of possibilities, using powerful tools such as RiTa.js, Google’s Cloud Vision API, and Tracery. We’ll learn how to tweet data from Google Spreadsheets, and to compose and tweet a song for an audio bot. We’ll also learn how to host our bot logic on Heroku and Docker. 

Twitter API Client for node

Supports both the REST and Streaming API.

twit

Create your own twitter bots

In this course, you will learn the basics of using Elasticsearch within your application. You will learn how to store and retrieve data in Elasticsearch as well as how to leverage its powerful search features.
 
You will be able to design indices, types and maps for your application to leverage the most from Elasticsearch and you will learn how to monitor the health and performance of your Elasticsearch servers to identify and resolve capacity issues before they impact your application.

Throughout this course, we will be using the [Simpsons dataset](https://www.kaggle.com/wcukierski/the-simpsons-by-the-data) containing all of the Simpsons episodes, characters, and lines. It will be the foundation for learning to retrieve data from Elasticsearch, perform complex searches, and understand and implement best practices for storing data in Elasticsearch. 

elasticsearch

Get Started with Elasticsearch

[Lodash](https://lodash.com/docs/) is a collection of utilities that have become a staple of modern web development. This collection will show you how to re-create its most common methods using simple JavaScript techniques available in any modern web browser. In addition to re-building tools like `_.merge`, `_.get` and `_.debounce` from scratch you will also learn how to replace them completely in some cases with modern syntax such as [optional chaining](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining), [nullish coalescing](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing_operator) and [Object.fromEntries](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/fromEntries).

This collection should interest the following groups:
- Those who want to better understand common JavaScript patterns
- Engineers who want to reduce their dependence on Lodash (for performance or other reasons)
- Candidates preparing for a technical job interview (where it is common to be asked how to implement some of these methods).

JavaScript® (often shortened to JS) is a lightweight, interpreted, object-oriented language with first-class functions, most known as the scripting language for Web pages, but used in many non-browser environments as well such as node.js or Apache CouchDB. It is a prototype-based, multi-paradigm scripting language that is dynamic, and supports object-oriented, imperative, and functional programming styles.

javascript

Recreating Popular JavaScript Utility Methods from Lodash

This lesson will demonstrate how to recreate a simplified version of the popular [lodash.merge](https://lodash.com/docs/4.17.15#merge) method from scratch.

First we'll create a test file with two objects that we want to merge together and demonstrate the output after running them through lodash.merge , then we'll go about creating our own version of the function.

Our initial version will support only one `source` object to merge in, and will support only the subset of JavaScript syntax supported by the JSON spec, including the following types:
- `boolean`, `number`, `string`
- object literal
- array

After getting basic types to merge we'll show off two methods for merging arrays, either using [concat](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/concat) or recursively going through the children and merging them.

Finally we'll build our own `isObject` function to check for object literals and recursively apply our `mergeJSON` function to any object literals in our `source` object.

And last we'll add support for multiple `source` objects to be merged in using a `for...of` loop.


**Note**

The `mergeJSON` method we create approximates the `lodash.merge` functionality but does not seek to replace it. It may be useful for some use cases, but primarily it's meant to be a tool for understanding how to merge objects with recursion.

Build lodash.merge from Scratch

This lesson will demonstrate how to replace common uses of [lodash.get](https://lodash.com/docs/4.17.15#get) with [optional chaining](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining) syntax.

`_.get` is by far my favorite lodash method. I've been on so many projects where the results of a network call was a large and often unreliable blob of data. After countless attempts to reign in errors caused by trying to access non-existent properties I've come to rely on _.get as a simple & safe way to retrieve data from those objects. However, as is always the case in web development, times are changing and the tools we need to safely access properties of complex objects are now being built into the platform. [Optional chaining syntax](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining) gives us the power of _.get without several of the tradeoffs. It doesn't require an extra library to use, it plays nicely with TypeScript and it can be used when calling methods as well as when accessing properties.

This lesson covers 4 common use cases of `lodash.get` and converts them to using optional chaining syntax.

1. Nested Object and Array Access
2. String and Variable Property Access
3. Calling a Method on an Object that may Not Exist
4. Providing a Default Value when a Property does Not Exist 

---

 ### Resources

- [Nullish coalescing operator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing_operator)

Replace lodash.get with Optional Chaining Syntax

This lesson will demonstrate how to recreate a simplified version of the popular [lodash.get](https://lodash.com/docs/4.17.15#get) method from scratch.

Our function takes 3 parameters `obj`, `string` and `defaultValue`.

The primary technique we discuss is splitting the `string` on any kind of separator we might see such as `[`, `]`, or `.` and then filtering out any empty strings we get in our array. Once we have that array of path parts, we can loop through each part with a `for..of` loop then keep reaching into object until we either a) run out of parts to try 🎉 or b) hit a dead end (`null` or `undefined`). The entire function is very simple to write, but can be incredibly powerful when you're safely trying to access large and complex blobs of data that might be coming from a database or server response. 

Build lodash.get from Scratch

This lesson will demonstrate how to recreate a simplified version of the popular [lodash.debounce](https://lodash.com/docs/4.17.15#debounce) method from scratch. **I literally failed a job interview by messing up this question, so watch carefully!**

Debounce is an incredible tool most commonly used to prevent responding too quickly to user input that is in motion. For example, preventing excessive AJAX requests made while a user types into an autocomplete field or quickly responding to window resize events without bringing your browser to a halt.

Our approach is to build the simplest `debounce` method possible using a [setTimeout](https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout) which gets cleared each time the supplied function is called thereby delaying the execution of our function until input stops for a given amount of time.

The [Lodash implementation of debounce](https://lodash.com/docs/4.17.15#debounce) includes support for a number of options including `leading`, `maxWait`, and `trailing` which we have not covered in this video, but may in future lessons.

### Variations

Modern implementations of `setTimeout` include support for additional arguments which get passed into your function so an alternative to the approach used in this lesson would be `setTimeout(fn, wait, ...args)`

### ES5 Compatbility

If you are in an es5 environment skip the rest/spread syntax and use:

```
var args = Array.prototype.slice.call(arguments)
```

And later, in your setTimeout function:

```
fn.apply(null, args)
```

----

There are some really great resources out there to learn more about debounce, here are my two favorites:

- https://css-tricks.com/debouncing-throttling-explained-examples/
- http://benalman.com/projects/jquery-throttle-debounce-plugin/


Build lodash.debounce from Scratch

This lessons builds on [Build lodash.debounce from scratch](https://egghead.io/lessons/javascript-build-lodash-debounce-from-scratch) to add support for one of its more interesting options: `maxWait`. The `maxWait` option ensures that a debounced method is deferred no longer than the time specified. This is helpful if we need to respond to a long running interaction while it's still in progress. 

In this lesson we'll demonstrate how continuously typing into a debounced input field prevents us from logging any input at all. We'll add a `maxWait` option as a third parameter to our `debounce` function and set it up using an additional `setTimeout` call.

Essentially `maxWait` works by adding a second timer which will fire in the case that the normal debounce timer does not get called. We avoid unnecessarily triggering our debounced function by calling `clearTimeout` in both of our `setTimeout` functions to clear out whichever timer didn't just fire. We also need to set the `maxTimer` variable to `null` to tell our function it's safe to start a new maxWait timer each time either timer fires.


**Notes**

- `setTimeout` returns a timer ID not the actual timer object itself, which is why we have to `null` it out *after* we already call `clearTimeout`
- If you don't `null` out `maxTimer` in your original timer, the maxWait timer will stop triggering after the debounce timer is called the first time. I learned that one the hard way when I first recorded this video.


Extending debounce with a maxWait Option

`_.pick` and its sibling `_.omit` are popular utilities which apply filter-like functionality to objects. This lesson will demonstrate two different techniques for building the functionality found in [_.pick](https://lodash.com/docs/4.17.15#pick).

1. Using `for...of` to loop through the `keys` array
2. Using `for..in` to loop through the target object

This lesson will also cover how to emulate `_.pick`'s support for either an array or multiple arguments for the picked keys using [Rest params](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/rest_parameters) and [Array.prototype.flat()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/flat).


Build lodash.pick from Scratch

`_.omitBy` and its sibling `_.pickBy` are popular utilities which apply filter-like functionality to objects. Both of them accept a function to determine if a given entry in the object should be included in the output object. This lesson will demonstrate an approach to build both of these utilities from a single function using the combo of `Object.entries` and `Object.fromEntries`.

`Object.entries` converts an object into an array of key, value pairs. We couple that with the native Array filter method, to create a generate `filter()` method that works on objects. Once we've filtered our object we use `Object.fromEntries` to convert the array back into an object. This works great for both `omitBy` and `pickBy`.

**Note**

`Object.fromEntries` is part of the ES2019 spec and may need [a polyfill](https://github.com/ungap/from-entries) to work depending on your environment.

---

- [MDN Guide: Object.entries](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/entries)
- [MDN Guide: Object.fromEntries](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/fromEntries)



Build lodash.omitBy and lodash.pickBy with Object.fromEntries

We’ll get a real-time stream of tweets, and learn how to filter a stream by keyword, location, and user.

Use the Twitter Stream API with Twit.js

Closing and opening indices on Elasticsearch allow you to free up resources in your cluster when they aren’t needed, preventing you from scaling or growing your cluster to support unused indices. In this lesson, you will see how easy it is to do both and learn to do so on your own indices.

Close and open an Elasticsearch index

In this lesson, you will learn how to add new documents to the Elasticsearch data store using HTTP methods and the Elasticsearch API. You will also learn how Elasticsearch uniquely identifies each document and what happens when you attempt to create a document that already exists.

Add data to Elasticsearch

Elasticsearch provides several methods for using its powerful search features. In this lesson, you will be introduced to the `_search` endpoint, the datapoints returned in results, how to set a query timeout (and what a timeout *doesn’t* do), and how to search specific indices and types.

Search for data in Elasticsearch using the _search endpoint

Elasticsearch provides a powerful API over http for accessing its features. In this lesson, you will be introduced to the API and learn how to use it to get data from Elasticsearch using the browser and the popular command curl.
We will see how adding  query stream parameters can exclude or include the properties that we desire from our data.

Setup is specified in the `README.md` of the repository linked below.

Get data from Elasticsearch by id using http

By default, search results are limited to the top 10 results. In this lesson, you will learn how to change the pagination size, paginate through results, and you will learn about the performance implications of pagination on Elasticsearch.

Paginate through search results in Elasticsearch

Elasticsearch offers two API endpoints for updating documents. In this lesson you will learn how to use those endpoints to replace entire documents in the store and update partial information within a document.

Update data in Elasticsearch

Elasticsearch provides a full featured client for nodejs available via npm. In this lesson, you will learn to how install the client and use it to retrieve data from your Elasticsearch server.
We will walk through retrieving data through callbacks as well as using promises.

Get data from Elasticsearch by id using the Elasticsearch npm client

Simple searches can be done using query parameters, also known as “Search Lite” by Elasticsearch. You will learn how to perform query parameter searches in this lesson and also discover the upper limits of its usability in this lesson.

Search Elasticsearch using Query Parameters

Query DSL is based on JSON and allows for Leaf and Compound query clauses.  The Query DSL is what you will want to use to write your production queries.  It makes your queries more flexible, precise, easier to read and easier to debug.

Search for data in Elasticsearch using queryDSL language

Using Full Text Search is where the search capabilities of Elasticsearch start to really set it apart from databases. In this lesson, you will learn how to perform full text searches against your data, interpret the results, and understand how the relevance score is impacts your search results. If you are following the examples in this course, be sure to import the Simpsons episode scripts for this lesson by running `node utils/episode_scripts.js` from the git repo directory.

Topics

Instructors

Content Type