Tech tutorial about adding geo-search support into AWS Amplify ecosystem and demonstration of adding custom resolvers against DynamoDB and OpenSearch so it would both on web and mobile without using any additional services like Amazon Location Service.
If you are reading this blog on wide monitor (24" +), please reduce window width to make reading easier.
Environment details
I am using GraphQL transformer version 2. The versions of the used libraries were:
- AWS Amplify version was "aws-amplify": "^5.3.8",
- amplify --version was 11.0.3
- nodeJS version was v16.15.
Implementation
Add new model into schema.graphql
First, you need to define new query / mutation in the GraphQL. Let's support searchByRadiusKm, and searchByBoundingBox.
Now, let's deploy the schema. Because we deployed model with @searchable attribute, the initial deployment would take about 15 minutes, since Amplify CLI would create new OpenSearch / ElasticSearch cluster. (reminder: there are associated costs with that after you would run out of the Free Tier).
Update ElasticSearch / OpenSearch index mapping to support geo queries
Now, the funny part. Using schema above, we now need to tell to the OpenSearch cluster, that Post.location field has special (Opensearch internal) type: "geo_point". In other words, we need to update mapping for that OpenSearch index in the OpenSearch console.
Important: You would need to do that when the index is empty and you would not be able to change mapping of the index later in the future - to update mapping of the index with the data, you would need to create new index and somehow add data to that.
To update mapping in the ElastichSearch manually, you would need to access ElasticSearch / OpenSearch console. The easiest way would be to add Access Policy, that allows access to the OpenSearch from your local IP address.
The Access Policy could look like this (just use correct name and your own IP address):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "YOUR_FULL_DOMAIN_ARN/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": "YOU.RI.PA.DDRESS/32"
}
}
}
]
}After saving that policy, you should be able to access OpenSearch Dashboard by clicking on OpenSearch Dashboards URL.
After opening it, go to the Dev Tools tab.
Now, if the index already contains some data, you can delete it by running commands on line 24 and then you can create new index by running command command from line 25. (to run command, just click on that line and click on play button).
Btw, the naming of the index is straightforward - the name of the index would be the name of the entity that encapsulates data type that should indicate geo coordinates.
Now, you can update mapping for Post, by running command on line 28. You would get response like this:
If you want quickly double-confirm that everything is correct, go to the Stack Management / Index Patters / indexName to check that location (field containing lat and lon fields) is set to be the geo_point.
Automated Index Mapping For Elastic/OpenSearch Index
Of course, all above was just manual action. But if you want to automatize that, you can use this Github Gist that is doing the same thing, right after the deployment of the OpenSearch cluster.
Add New Custom Resource into Amplify project (CDK Introduction)
After updating mapping in OpenSearch/ElasticSearch, we will need to add new custom resource into the project.
# run this command to create new resource
amplify add custom
# then choose AWS CDK
How do you want to define this custom resource? … (Use arrow keys or type to filter)
❯ AWS CDK
AWS CloudFormationAfter running that command, you will have few files added into project where the most important ones will be cdk-stack and package.js that is laying next to that.
The custom resource will be created at following path:
rootFolder/amplify/backend/custom/nameOfYourCustomResource
The freshly created CDK stack would look like this:
import * as AmplifyHelpers from "@aws-amplify/cli-extensibility-helper";
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import { AmplifyDependentResourcesAttributes } from "../../types/amplify-dependent-resources-ref";
export class cdkStack extends cdk.Stack {
constructor(
scope: Construct,
id: string,
props?: cdk.StackProps,
amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps
) {
super(scope, id, props);
/* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
new cdk.CfnParameter(this, "env", {
type: "String",
description: "Current Amplify CLI env name",
});
const { envName, projectName } = AmplifyHelpers.getProjectInfo();
}
}
Note: In my case, I also needed to change version of the "aws-cdk-lib" to be "~2.80.0" in the package.json for the custom resource due to the Typescript errors. In then end, my package.json dependencies for custom resource were "@aws-amplify/cli-extensibility-helper": "^3.0.0", "aws-cdk-lib": "~2.80.0", "constructs": "^10.0.5".
Adding New Custom JS Resolvers With CDK in AWS Amplify Project
Fortunately, the naming of AWS Amplify resources is well-thought and relies on properties that are easily accessible - such as GraphQL (AppSync) API name.
In regard to the CDK itself, it uses 2 syntaxes (L1 and L2) and it compiles down to the Cloudformation scripts.
Simply, if you want to create new custom resolver, you can use either cdk.aws_appsync.Resolver or cdk.aws_appsync.CfnResolver object and just create new objects. Deployment will of those resources will stay the same (amplify push --y) command.
In the CDK example below, you will see 3 custom JS resolvers showing:
- how to support DynamoDB BatchGetItems / Transactions with Appsync
- how to create JS resolver against OpenSearch for searching by distance (radius)
- how to create JS resolver against OpenSearch for searching by bounding box
import * as AmplifyHelpers from "@aws-amplify/cli-extensibility-helper";
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import { AmplifyDependentResourcesAttributes } from "../../types/amplify-dependent-resources-ref";
export class cdkStack extends cdk.Stack {
constructor(
scope: Construct,
id: string,
props?: cdk.StackProps,
amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps
) {
super(scope, id, props);
/* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
new cdk.CfnParameter(this, "env", {
type: "String",
description: "Current Amplify CLI env name",
});
const { envName,projectName } = AmplifyHelpers.getProjectInfo();
const dependencies: AmplifyDependentResourcesAttributes = AmplifyHelpers.addResourceDependency(
this,
amplifyResourceProps.category,
amplifyResourceProps.resourceName,
[
{
category: "api",
resourceName: "lake", // <- Adjust with name of your API resource
},
]
);
// Get the ID of the existing GraphQL API
const apiId = cdk.Fn.ref(
dependencies.api.lake.GraphQLAPIIdOutput // <- Adjust with name of your API resource
);
// References the existing API via its ID
const api = cdk.aws_appsync.GraphqlApi.fromGraphqlApiAttributes(this, "API", {
graphqlApiId: apiId,
});
const requestJS = `
import { util } from '@aws-appsync/utils'
export function request(ctx) {
return {
operation: 'GetItem',
key: util.dynamodb.toMapValues({ id: ctx.args.id }),
}
}
export function response(ctx) {
return {
body: ctx.result,
text: 'Hello from JS resolver'
}
}
`;
const dynamoDataSource = api.addDynamoDbDataSource(
"dynamo-datasource",
cdk.aws_dynamodb.Table.fromTableName(this, "tableTest", `Post-${apiId}-${envName}`)
);
const resolver = dynamoDataSource.createResolver("ExecuteMutationResolver", {
typeName: "Mutation",
fieldName: "executeMeResolver",
runtime: new cdk.aws_appsync.FunctionRuntime(
cdk.aws_appsync.FunctionRuntimeFamily.JS,
cdk.aws_appsync.FunctionRuntime.JS_1_0_0.version
),
code: cdk.aws_appsync.Code.fromInline(requestJS),
});
new cdk.aws_appsync.CfnResolver(this, "ExecuteOpenSearchMutationResolver", {
apiId: apiId,
fieldName: "searchByRadiusKm",
typeName: "Mutation",
dataSourceName: "OpenSearchDataSource",
runtime: {
name: cdk.aws_appsync.FunctionRuntimeFamily.JS,
runtimeVersion: cdk.aws_appsync.FunctionRuntime.JS_1_0_0.version,
},
code: `
import { util } from '@aws-appsync/utils'
export function request(ctx) {
if(!ctx.identity) {
util.unauthorized()
}
const latitude = ctx.args.originLatitude;
const longitude = ctx.args.originLongitude;
const radius = ctx.args.radiusKm;
return {
operation: "GET",
path: "/post/_search",
params: {
headers: {},
queryString: {},
body: {
from: 0,
size: 50,
query: {
bool: {
must: {
match_all: {}
},
filter: {
geo_distance: {
distance: radius + "km",
location: {
lat: latitude,
lon: longitude
}
}
}
}
}
}
}
};
}
export function response(ctx) {
const { result, error } = ctx;
if (error) {
util.error(error.message, error.type)
}
return ctx.result.hits.hits.map((hit) => hit._source);
}
`,
});
new cdk.aws_appsync.CfnResolver(this, "ExecuteOpenSearchMutationResolverBB", {
apiId: apiId,
fieldName: "searchByBoundingBox",
typeName: "Mutation",
dataSourceName: "OpenSearchDataSource",
runtime: {
name: cdk.aws_appsync.FunctionRuntimeFamily.JS,
runtimeVersion: cdk.aws_appsync.FunctionRuntime.JS_1_0_0.version,
},
code: `
import { util } from '@aws-appsync/utils'
export function request(ctx) {
if(!ctx.identity) {
util.unauthorized()
}
const latitudeTopLeft = ctx.args.topLeftLatitudePoint;
const longitudeTopLeft = ctx.args.topLeftLongitudePoint;
const latitudeBottomRight = ctx.args.bottomRightLatitudePoint;
const longitudeBottomRight = ctx.args.bottomRightLongitudePoint;
return {
operation: "GET",
path: "/post/_search",
params: {
headers: {},
queryString: {},
body: {
from: 0,
size: 50,
query: {
bool: {
must: {
match_all: {}
},
filter: {
geo_bounding_box: {
location: {
"top_left": {
"lat": latitudeTopLeft,
"lon": longitudeTopLeft
},
"bottom_right": {
"lat": latitudeBottomRight,
"lon": longitudeBottomRight
}
}
}
}
}
}
}
}
};
}
export function response(ctx) {
const { result, error } = ctx;
if (error) {
util.error(error.message, error.type)
}
return ctx.result.hits.hits.map((hit) => hit._source);
}
`,
});
}
}
Explanation:
- We need to get Appsync apiId (lines 22-37) to use it for adding new resolvers for corresponding environment.
- DynamoDB - if you need, you can create new datasource that could theoretically reference other DynamoDB table deployed outside of AWS Amplify framework (lines 62-65)
- DynamoDB- shows Sreating new resolver against DynamoDB with "Inline" resolver's code. You can use that approach to add support for retrieving data in Batches, or using Transactional updates. When creating new resolver, it's important to refer to the mutations defined in your schema.grapqhl (in my case typeName: "Mutation" and fieldName: "executeMeResolver").
- OpenSearch - deals with creating new custom JS resolver with CDK in AWS Amplify project for OpenSearch - (lines 77-209). Notice, how you can access the arguments passed from Appsync, how it's mapped to the OpenSearch request that uses proper index name and how we are passing radius to the distance query. If you need more filtering to be made, you would need to update bool.must.match_all object (e.g. replace it with the other OpenSearch filter queries in the pic below).
- How it should exactly look like, you can find in the OpenSearch console that you should now have access to.
- OpenSearch - To support search by bounding box (rectangular search), you would need to pass at least 2 parameters instead of just radius: top left point, and bottom right point.
Note: In the first example of adding JS Resolver for OpenSearch, filter.geo_distance.location is the name of the field with type geo_point.
Results
As you can see, now you can search by distance or use rectangular search within AWS Amplify app. If it would be (ever) needed, you should know everything to :
- add new resolvers e.g. to support searching in the "polygon-area"
- add more features for the DynamoDB-backed resources such as transactions or batch gets.
- implement more robust authentication check
Credits
Thank you goes to the all participants in this Github thread: https://github.com/aws-amplify/amplify-cli/issues/13183. Those we were visible there in the moment of writing this article were:
