Amazon SQS offers fully managed FIFO / standard queues. Your service(s) may want to add items/jobs/messages to a queue and process them later. This guide will show you steps to create and access Amazon SQS queues from your services.

​
Declarative & Automatic using ops.json:

For your service, if you need one or more SQS queues, you can add ops.json in the root directory of Github repo you’ve connected for the service.

Learn more about configuring dependencies using ops.json here.

And add SQS queues as a dependency, like below.

{
  "dependencies": {
    "sqs": {
      "queues": [
        {
          "id": "test123",
          "prefix": "testdep123",
          "fifo": true,
          "exports": {
            "MY_SQS_QUEUE_NAME1": "$name",
            "MY_SQS_QUEUE_ARN1": "$arn"
          }
        }
      ]
    }
  }
}

You can add as many queues as you want.

Arguments you can add in each queue object above:

  1. id - Alphanumeric string. Must be unique amongst the queues you’ve declared above. Changing this string will replace the original queue with new one.
  2. prefix - Alphanumeric string. Will be used as a prefix in the name of your queue. Changing this string will replace the original queue with new queue.
  3. fifo - Set this to true if you need a FIFO queue. Set it to false or leave out the fifo key if you want a standard queue instead.
  4. exports - Set of key value pairs. Keys are the ENVIRONMENT VARS we will pass to your code / containers. Values are the properties of the queue provisioned. See below for available properties.
Changing id or prefix string will replace the original queue with new one.

Available properties of the queue to use in exports:

  1. $name - Name of the queue
  2. $arn - Amazon resource name of the queue. Eg., arn:sqs:..

​
Lifecycle:

If you have provided ops.json at the root of the git repository, it will be processed if a corresponding service in any of your active environment points at the same repository as source and when a deployment is triggered.

  • When the service is spinned up first time or when a new deployment is triggered, ops.json is parsed for processing. Resources declared in the dependencies object will be provisioned before your code starts to run.
  • Resources with same id are provisioned only once for the life of the service. And updated when there is a change in one of the properties above.
  • Keys in exports object will be passed as enviroment variables to your service.
  • When the service is deleted, the provisioned queues are deleted from the cloud account immediately & automatically.

​
Access with no keys:

Queues can be accessed from your code using AWS SDK. You don’t have to pass any AWS secret key to authenticate. Your containers automatically authenticate to have access to all the queues you’ve declared in the above JSON.

LocalOps arranges Role based access by attaching an IAM role for all the services run within your environment. We provision a policy like below on that IAM role for every queue provisioned.

{
			"Version": "2012-10-17",
			"Statement": [
				{
				"Effect": "Allow",
				"Action": [
					"sqs:SendMessage",
					"sqs:ReceiveMessage",
					"sqs:DeleteMessage",
					"sqs:GetQueueAttributes",
					"sqs:GetQueueUrl",
					"sqs:ChangeMessageVisibility"
				],
				"Resource": "your-queue"
				}
			]
	}

Curious to know the IAM role that is used? Visit your environment’s main dashboard in LocalOps console to see the role.

​
Manual provisioning of new SQS queues:

To proceed with this guide, you would need access to:

  1. AWS console with permissions to create SQS queues and add policies to an existing IAM role.
  2. LocalOps console

​
1. Create SQS queue

Login to AWS console and create new SQS queue in the region you want, and with appopriate unique name.

You can follow this official AWS guide to create a sqs queue.

​
2. Add permission to access SQS queue

LocalOps creates a new IAM role in your AWS account for your services/containers to use. So that your code/containers can access any AWS resource without passing AWS_ACCESS_KEY or AWS_SECRET_KEY.

Navigate to LocalOps console and the corresponding environment. In overview section, you can find the APP ROLE whose name ends with app-role and that which you can copy and search in AWS IAM console > Roles section. Once you find the IAM role in IAM console, next step is to add a policy to the IAM role to give permissions for the role to access the SQS queue your created.

​
Create a new IAM policy for the SQS queue:

  1. Navigate to Policies in AWS IAM console
  2. Create a new IAM policy and give it a specific name
  3. Add the JSON below, as policy statement:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sqs:SendMessage",
        "sqs:ReceiveMessage",
        "sqs:DeleteMessage",
        "sqs:GetQueueAttributes",
        "sqs:GetQueueUrl",
        "sqs:ChangeMessageVisibility"
      ],
      "Resource": "<queue-arn>"
    }
  ]
}

Replace <queue-arn> with your SQS queue’s ARN.

If you create more than one queue, expand the Resource attribute above to an array of ARN strings.

  1. Press save.

​
Add the new IAM policy to the IAM role

Navigate back to the IAM role you saw earlier. In its Permissions tab, pick the option to “Add Policy”. In the policy picker, pick the one you created above and add it.

​
3. Add queue details as secret

Last step is to give service access to the new SQS queue. In LocalOps console, navigate to the Service settings within the corresponding environment.

In secrets section, add a new key value pair like

- Key: `queue-var-name`
- Value: `queue-name`

Repeat this for each queue you created. Learn more about secrets here.

You can name the key in any way you want. In your code, you can access queue-var-name environment variable to get the name of the queue you created. This can be passed to the AWS SDK used in your code to read/write data. There is no need to provide credentials like access key since we are having role based access control here.

That’s it! 🎉