Ramping up on the AWS Amplify CLI code base
Introduction
Decoding magic can be hard but not impossible. Take a deep breath before I start with a nice overview of the Amplify CLI and its source code. Be patient as I unfold the mysteries behind this large code base.
Amplify framework provides multiple products to build full stack iOS, Android, Flutter, Web, and React Native apps. Amplify CLI helps you to configure all the services needed to power your backend through a simple command line interface. Amplify CLI depends on various Amazon Web Services like AppSync for data modelling and GraphQL API’s, CloudFormation templates for Infrastructure as Code and creating replicable stacks and a lot more to provide functionalities like authentication, storage, hosting, functions and REST API’s.
Amplify CLI Architecture
Now you know why everyone was talking about Amplify on Twitter. Lets jump to the real part and try to understand how this beast framework works under the hood.
The source code can be overwhelming at first. It is a monorepo, a multi package repository written in TypeScript and JavaScript and managed by Lerna. A monorepo makes it easier to share and refactor code. I assume that you are ready to dive deep with an instance of amplify-dev running on your system after following the local environment setup instructions. The official AWS Amplify docs are your best friend and I will list down some specific sections in the docs which can give a better understanding of the working instead of opening and exploring random files in the codebase.
Amplify CLI has a pluggable architecture where the core provides the pluggable platform and most of the CLI category functions are implemented as plugins.
There are 4 main types of plugins:-
- Category: Holds logic to create and manage one category of backend resources in the cloud.
- Provider: Abstracts the actual cloud resource provider and exposes methods for the category plugins to CRUD cloud resources. Currently, the only official provider plugin,
amplify-provider-awscloudformation, uses AWS CloudFormation to create backend resources in AWS. - Frontend: Handles specific types of frontend projects, such as JavaScript, Android or iOS projects.
- Util: Provides certain CLI commands and functionalities for the CLI core, and other plugins.
Apart from the 4 main plugin types the Amplify CLI project holds the code for its core in the packages/amplify-cli and packages/amplify-cli-core directories. The packages/amplify-cli is a plugin of type core and holds code for some of the most used commands like init (used to start a Amplify project), push, pull, publish, status etc. You can checkout the full list of commands available in packages/amplify-cli/amplify-plugin.json . Plugins communicate with the CLI core and with each other through the project metadata. The CLI core provides the read and write access to the project metadata for the plugins. The project metadata is stored in the file amplify/backend/amplify-meta.json in the user project.
You can explore the plugin package directory structure in detail here.
Amplify CLI uses AWS CloudFormation and nested stacks to create reusable templates and replicable stacks to simplify infrastructure management.
Amplify CLI uses CircleCI for continuous integration. 3 main workflows are used to run integration tests, do resource cleanups and build, test and deploy a new GitHub release.
Amplify CLI has a base TypeScript configuration in the root tsconfig.base.json file. Many plugins in the packages directory extend this base TypeScript configuration and hold the source files in the src folder and all the emitted/compiled files in the lib folder.
Integration tests for Amplify CLI are present in the amplify-e2e-core and amplify-e2e-tests packages. These tests use nexpect to run the CLI. It allows chaining so that we can compose complex terminal interactions. The amplify-e2e-core contains code shared between amplify-e2e-tests package and holds helper methods that can help you create resources and setup and delete projects.
Some packages like the graphql-key-transformer also use snapshot testing with Jest. A typical snapshot test case renders a UI component, takes a snapshot, then compares it to a reference snapshot file stored alongside the test. The test will fail if the two snapshots do not match: either the change is unexpected, or the reference snapshot needs to be updated to the new version of the UI component. However, snapshots can capture any serializable value and should be used anytime the goal is testing whether the output is correct. You can refer to the Jest repo to see an example of snapshotting CLI output. You can read more about snapshot testing in this section of Jest docs.
The graphql-transformer in Amplify is pure magic and helps you build, deploy and maintain GraphQL API’s really fast. It takes as input a GraphQL SDL document and a list of GraphQL transformers and returns a CloudFormation document that fully implements the data model defined by the input schema. Read this section to understand the transform lifecycle in detail.
My Experience
I was new to AWS when I started contributing to this huge project so I can understand the pain and confusion. Whenever you come across a new service name in the docs say AWS CloudFormation for example you can do a quick search for the same and read about it from the official AWS docs for that service. You do not have to read the complete docs for a service. Just skim through the sections which describe what the service does and how it works and you will feel ready and confident to move on. You can always learn more about specific parts if you choose to work on an issue related to that service.
The first phase of contributing to any open source project is to become a good user of that project. In this case you can explore the functionalities of the Amplify framework by making a small sample project with React by following this guide.
The CLI places a folder structure in the root directory of the sample project during amplify init . You can read this section to know more about the contents stored in these files.
Now that you have made a sample project its time to understand different parts of the Amplify codebase so that you can solve issues. I along with my awesome teammates Raj, Gita and Pawan came up with a detailed set of guides called Amplify CLI Monorepo tour which will help you understand different parts of the project. You are welcome to contribute to this project here and make it even better for other new contributors and Amazon engineers working on AWS Amplify CLI. Here is a deck of slides which explains more on the motivation behind this project.
Now try to pick up issues labelled as good first issues from the Amplify CLI GitHub repository. Try to reproduce it using the sample project you made with the Amplify CLI. Do not forget to join the awesome community of developers and contributors on AWS Amplify Discord. There are channels for first time contributors and CLI discussions where you can get help from the Amazon Amplify staff to make your first contributions.
Now I assume that you managed to reproduce an issue and the bug is right in front of your eyes. It is time to debug and identify which part of this large codebase it lies in and submit a patch.
Debugging in AWS Amplify CLI with VS Code
VS Code has built-in debugging support for the Node.js runtime and can debug JavaScript, TypeScript, or any other language that gets transpiled to JavaScript.
To run or debug a simple app in VS Code, select Run and Debug on the Debug start view or press F5 and VS Code will try to run your currently active file.
However, for most debugging scenarios, creating a launch configuration file is beneficial because it allows you to configure and save debugging setup details. VS Code keeps debugging configuration information in a launch.json file located in the.vscode folder.
To create a launch.json file, click the create a launch.json file link in the Run start view in the Amplify CLI codebase.
VS Code will try to automatically detect your debug environment, but if this fails, you will have to choose it manually.
Here is the launch configuration generated for Node.js debugging.
Note that the attributes available in launch configurations vary from debugger to debugger. Update the contents of the launch.json as given below to connect it with your Amplify CLI sample project.
Let us understand how you can modify this launch.json to debug any part of the project. In the above example the cwd key holds the absolute path to the working directory of the program being debugged. You can update this value with the path to your sample project you created using Amplify CLI. You can change the name of the configuration with the name key depending on the part of project you are trying to debug. Here I was trying to debug the gql-compile command in the api category. This name will appear in the launch configuration dropdown menu. The args key holds the command line arguments passed to your program. I wanted to debug the flow of the command amplify api gql-compile so I passed 2 arguments api and gql-compile . For context the api gql-compile command is used to compile the backend/api/~apiname~/schema.graphql . The compiled schema output is generated in backend/api/~apiname~/build/schema.graphql .
In VS Code, there are two core debugging modes, Launch and Attach, which handle two different workflows and segments of developers. We are using a launch configuration here. You can think of a launch configuration as a recipe for how to start your app in debug mode before VS Code attaches to it, while an attach configuration is a recipe for how to connect VS Code’s debugger to an app or process that’s already running. The request key in the launch.json file specifies the type of configuration.
Now that you are ready with the debugger configuration you can add breakpoints in relevant files by clicking on the editor margin. Breakpoints in the editor margin are normally shown as red filled circles. You can also add logpoints which is a variant of a breakpoint that does not “break” into the debugger but instead logs a message to the console. Logpoints are especially useful for injecting logging while debugging production servers that cannot be paused or stopped. Log messages are plain text but can include expressions to be evaluated within curly braces (‘{}’).
You can read more about the different types of breakpoints you can add in VS Code here. Happy debugging!
Conclusion
I worked with the Amazon team on the AWS Amplify project as a part of the MLH Open Source program. This project has a huge scope of learning and I still feel that a lot is left to be done. I had a really nice experience working with highly experienced software developers, developer advocates and the Amplify product manager in the Amazon team. We as a team of 4 students used to meet weekly with the Amazon team along with our technical mentor who really helped us in debugging and understanding different parts of Amplify. We could also schedule career advice and pair programming sessions with different experienced technical mentors via the Raise.dev platform. No doubt MLH is one of the best learning experience for students.
I hope I was able to motivate you and help you get started with the Amplify CLI codebase. Take your time to get familiar with different parts. You can start by exploring the init workflow which is used to setup and initiliaze an Amplify project. Fret not, you’ll ramp up in a while. After all it takes time to decode magic. Keep it going. I’ll meet you soon with another tech blog and a new set of learnings. Cheers.
