AWS Transfer for SFTP

Info: This is our first post of a serie of coauthoring articles with @kharec

Serverless SFTP with AWS Transfer for SFTP

Sometimes in web world, we need to quickly deploy a space to share datas with programs or with other humans.

One of the numerous solutions is an SFTP space out there. But you know: create the server, configure the service, partition the users, the permissions, the folders, etc… It’s getting heavy faster than a speeding bullet!

So at Amazon Web Services, they implemented a service to create SFTP server/account, quickly and easily, backed onto an S3 bucket. And the beauty of it : no servers to manage at all !

Today, we present you AWS Transfer for SFTP !

Why using SFTP on AWS

For enterprise ETL systems, batch processing, legacy use cases. Main advantages of this managed service solution are: built-in auto-scaling capabilities, native high availability without managing a single server, patching, OS maintenance, networking, and security.

Automated Deployment

Using AWS CloudFormation

Deploying your server:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


AWSTemplateFormatVersion: 2010-09-09
Description: SFTP CloudyMinds

Resources:
  SFTP:
    Type: AWS::Transfer::Server
    Properties:
      EndpointType: PUBLIC
      Tags:
        - Key: Name
          Value: CloudyMinds-SFTP

Using Terraform

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49


resource "aws_iam_role" "foo" {
  name = "tf-test-transfer-server-iam-role"

  assume_role_policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
        "Effect": "Allow",
        "Principal": {
            "Service": "transfer.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
        }
    ]
}
EOF
}

resource "aws_iam_role_policy" "foo" {
  name = "tf-test-transfer-server-iam-policy-%s"
  role = "${aws_iam_role.foo.id}"

  policy = <<POLICY
{
    "Version": "2012-10-17",
    "Statement": [
        {
        "Sid": "AllowFullAccesstoCloudWatchLogs",
        "Effect": "Allow",
        "Action": [
            "logs:*"
        ],
        "Resource": "*"
        }
    ]
}
POLICY
}

resource "aws_transfer_server" "foo" {
  identity_provider_type = "SERVICE_MANAGED"
  logging_role           = "${aws_iam_role.foo.arn}"

  tags = {
    NAME = "tf-acc-test-transfer-server"
    ENV  = "test"
  }
}

Terraform Documentation

Manual deployment (Using Console)

Connect to AWS Console
Go to AWS Transfer for SFTP Service
Create Server
Basic configuration:
- Public or Private
- Customize your endpoint DNS name
- Identity Provider: Internal or using API Gateway (custom) Using Internal, the user base will be managed by the service (!= IAM users)

Using awscli

You can go from nothing to a fully functional SFTP server, users included, with the formidable awscli. In this article, we suppose that you already installed it. So, first things first, let’s create the server.

1
2
3
4


$ aws transfer create-server
{
    "ServerId": "s-2232098796514635a"
}

Once it’s created and online, we can get all its information via the following command :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


$ aws transfer describe-server --server-id s-2232098796514635a
{
    "Server": {
        "Arn": "arn:aws:transfer:eu-west-1:829937339934:server/s-2232098796514635a",
        "IdentityProviderType": "SERVICE_MANAGED",
        "ServerId": "s-2232098796514635a",
        "State": "ONLINE",
        "Tags": [],
        "UserCount": 0
    }
}

We get an endpoint as well and we can see it in the console :

AWS Transfer endpoint

Let’s try our endpoint :

1
2
3
4
5


telnet s-2232098796514635a.server.transfer.eu-west-1.amazonaws.com 22
Trying 18.200.52.219...
Connected to s-2232098796514635a.server.transfer.eu-west-1.amazonaws.com.
Escape character is '^]'.
SSH-2.0-OpenSSH_7.7

OK, it’s up.

Now let’s create the user to connect and exploit our SFTP server. Note that we have created an IAM Role that has been applied with the policy AmazonS3FullAccess for the process. To create the user, we provide the ARN of this role.

1
2
3
4
5


$ aws transfer create-user --server-id=s-2232098796514635a --user-name=kharec --role arn:aws:iam::829937339934:role/awstransfer
{
    "ServerId": "s-2232098796514635a",
    "UserName": "kharec"
}

Now, to access our SFTP, we have to authenticate with an ssh key, as AWS Transfer doesn’t support the password authentication. To do that, we need to import our public key to this server and for this user.

1
2
3
4
5
6


$ aws transfer import-ssh-public-key --server-id=s-2232098796514635a --user-name=kharec --ssh-public-key-body "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCRGvlEWZJJf05P+cdIZqJk0xSvdt5ABaoDDiKVrN4LzXCcI3ayzaLGZOmGozkiT+6CuW+h9RJblmVhl8S/s19Yrynx3YSV9bUBy6LTiinsTI11xQfMVyCOQcjvETpJVF0koy2YVgTtLPTdX1VWqyN2+Q/MtIx/0YXKHzmDRXw3rTSJQOLcgho48JY/Lm8YYpgMVOC7PUISPAmU1CqnG9yZLhFalPuymCnZTbVBhmevN4nHcV3AKOrKY6wq4s/AJyWgX9b2JEghwKpXMdALaPAuWA6B6/d2p2Yd7UMj1T1NLlHr9friLntMrk+kZs9K3i6JwRlGT1WIAEq2nRg79Ws1 kharec@matty"
{
    "ServerId": "s-2232098796514635a",
    "SshPublicKeyId": "key-71188869ef65466c8",
    "UserName": "kharec"
}

Easy ! Finally, let’s try and connect:

1
2
3
4


$ sftp kharec@s-2232098796514635a.server.transfer.eu-west-1.amazonaws.com 22
Warning: Permanently added the RSA host key for IP address '108.129.43.116' to the list of known hosts.
Connected to kharec@s-2232098796514635a.server.transfer.eu-west-1.amazonaws.com.
sftp>

Pricing

You will be charged for two types of usage on Transfer for SFTP:

Time your SFTP endpoint is provisioned: 0,30$ per hour
Data Transfer (IN and OUT) $0.04 per gigabyte (GB) transferred

As you may already know, on AWS, the data transfer IN is usually free, this is not the case with this AWS Managed Service.

Example:

full-time SFTP Endpoint: $216 monthly
10GB uploaded: $12 monthly
Monthly total: $228

Source: AWS Transfer for SFTP - Pricing - Give a try on the pricing calculator embedded on this page.

Challengers

Any type of instance in any cloud (GCP, Azure, Digital Ocean) with an SFTP server installed on it.
AWS S3 bucket
Azure Storage blobs
Google Cloud Storage

Known limitations

No user quota

You have to know that you cannot apply quotas on an SFTP user. So he can upload as much as he wants. It can be a weakness in you bill.

No user chroot

If you have more than one user on your SFTP endpoint, you have to know that you cannot restrict your users into a directory as you can with OpenSSH.

No filetype filtering

In AWS Transfer, you cannot apply some policy to restrict the filetypes sent. It can be a security problem, your users can upload any type of files.

No IP restrictions

You cannot apply a security group or equivalent to restrict your SFTP endpoint by IP. Even if the authentication is key-based, it can be an issue.

Alternatives

In case SFTP is not a mandatory protocol for you, you can use cheaper options:

AWS CLI (aws s3 sync, aws s3 cp, aws s3 presign)
S3 API backed Software (S3 Browser, Cloudberry, Cyberduck, Filezilla Pro, …)
S3 Console with least privilege rights for your users

We hope you enjoyed this article. If you want to know more, feel free to discuss it with us on social networks or by email !

kharec & zoph

That’s all folks!

zoph.

Why using SFTP on AWS#

Automated Deployment#

Using AWS CloudFormation#

Using Terraform#

Manual deployment (Using Console)#

Using awscli#

Pricing#

Challengers#

Known limitations#

No user quota#

No user chroot#

No filetype filtering#

No IP restrictions#

Alternatives#