Full Stack Machine learning on AWS: Reading Text from an Image with Amplify & Rekognition

Nader Dabit: [0:01] We'll start by creating a new React application by running npx create-react-app. Next, we'll change into the new directory and install aws-amplify using either NPM or Yarn.

[0:22] To create the new Amplify project, we can run amplify init. Here, we'll give the project a name, the environment a name, choose our default text editor, and then choose the default for the rest of the options.

[0:40] When prompted for our AWS profile, we'll choose the AWS profile that we'd like to use.

[0:50] The Predictions category for machine learning has a dependency on auth being enabled. We'll next enable auth by running amplify add auth and accepting all of the default options.

[1:06] Next, we can add Predictions by running amplify add predictions. Here, we'll choose Identify as the category type and then Identify Text for what we'd like to identify. We'll then give a name for the resource and choose yes to identify documents.

[1:24] When prompted for who should have access, we'll choose auth and guest users, because we would like unauthenticated users to be able to use our app.

[1:34] To deploy the services, we can now run amplify push. Now that the services have been deployed, we'll open up our text editor and then open source/index.js to configure our project to work with Amplify.

[1:49] In this file, we'll import Amplify from the aws-amplify library, the configuration from aws-exports, and then we'll also import the AmazonAIPredictionsProvider from aws-amplify/predictions.

[2:14] To configure the Amplify project, we can call amplify.configure passing in the config. To enable Predictions, we'll also need to call amplify.addPluggable passing in a new instance of the AmazonAIPredictionsProvider.

[2:35] Next, we can open App.js. We'll need to manage some component states so we'll first import useState from React. We'll next import the Predictions API from aws-amplify.

[2:55] To hold the response from the API, we'll create a variable called response and a function called setResponse by using the useState hook from React. Next, we'll create a function called identify to call the Predictions API.

[3:18] When this function is invoked, we'll first update setResponse to let the user know that we are in the process of identifying the text. Next, we'll destructure the files coming off of the event that will be sent in from the users input. We'll then create a variable called file taking in the first file within the files array.

[3:44] Next we'll call Predictions.identify setting text as the Predictions type. For the source, we'll pass in the file that we destructured off of the event. We'll then set the format to PLAIN. The options for formatting are PLAIN, FORM, TABLE, and ALL.

[4:10] Once the response comes back from Predictions.identify, we'll then set the response by calling setResponse passing in the data.text.fullText.

[4:29] In the user interface, we'll now create a title called Text identification. We'll then create a file input and set the onChange handler to invoke the identify function.

[4:53] Finally, we'll render the response within a paragraph tag. Now we're done, and we should be able to go ahead and test this out. To do so, we'll open the command line and run npm start.

[5:07] To get some text to identify, I'll open an article on Medium and take a screenshot.

[5:21] When we upload an image, the API should read the text from the image, return it in the response, and then our React app should render it to the screen.

[5:34] Next, let's try one more image using something a little different.