1 02 2016
Openshift registry on Amazon S3
Few weeks ago, I was working with Openshift on AWS. For this reason I was looking for a cool feature : Use an AWS s3 bucket as backend for Openshift registry.
This feature works but it’s not very documented yet. In this post we are going to share with you the issues we encountered and how we did to make it working.
All the following steps have been tested on Openshift v220.127.116.11.
Create the registry
The first step is to create a dedicated IAM for the S3 and apply this kind of policy on your bucket:
Once the policy is set, it’s time to create the registry. We have to start with a basic one, after that we will apply the s3 config.
Openshift provide an Ansible playbook to configuration the registry with s3 backend. Let’s use it.
Test the registry on S3
For this part the best is to build a new app to test the full stack. In our case we only want to do some quick tests focused on the registry only.
Get the service IP of the registry:
The First dummy test is to curl the registry to know if the container listen at least on the expected port :
It’s time to do a more representative test. Manually push an image in the registry. (https://docs.openshift.com/enterprise/3.0/install_config/install/docker_registry.html#access)
Start to get a valid token from one of your Openshift users
Pull a small image for the test. For example the busybox image. Tag this image on our registry and push it :
If your configuration is ok, you should be able to find new directory and data in the s3 bucket
The first error we faced was a GO stack trace :
301 response missing Location header
It was due to a to a miss config of the s3 bucket region. The output message is not very user friendly. If you have some stack trace start to verify s3 bucket name, region and credentials.
tips : To ensure the S3 registry configuration applied to the registry is correct :
Ansible generate config file from this template (https://github.com/openshift/openshift-ansible/blob/master/playbooks/adhoc/s3_registry/s3_registry.j2). You can find this generated file in /root/config.yml during the execution of the playbook but also directly inside the registry container.
Just go into the registry container and verify the config file given at the docker registry daemon.
Example of our config file :
That’s it. Hoping this post can help someone.