Artifactory Usage

Hello all!

I have a quick note on how to use Artifactory repository and integrate with Bintray. This post is also a generic knowledge post.

In artifactory, when setting up a repository, you will have 3 types: local, remote and virtual. You might see distribution repo too, but it will be other type.

• Local: is a physical, "real" repository that is managed, reside in artifactory. This is usually a default deployment repository within a virtual repository
• Remote: is a caching proxy for a remote URL. This will help to reduce to bandwidth of downloading packages from public repos. However, in order to use this remote repo, you have to change the sources.list repo point to this remote repo on your artifactory.
• Virtual: is a group of repositories. This is a great practice of using virtual repo to group different repos within it, the client usage will point to this virtual repo only.
• Distribution: is a set of rules, not a real repo contains packages, which will distribute the packages from artifactory to Bintray. This is a hub or interface for Bintray integration.

You can refer here to create a good naming convention for your repo on artifactory with their best practices.

Select the "Set Me Up" on each repo to get the REST API syntax for each package type. Please note:

• Remote repos are readonly, no deployment to these repos
• You should setup default deployment repo of a virtual repo to a local repo, so when you deploy to a virtual repo, this will actually deploy into your local one.
• Configure distribution repo so distribute to Bintray will be manipulate from aritfactory REST API

For debian package, to deploy (refer here for more tips on using debian repo on artifactory)

curl -H 'X-JFrog-Art-Api:<api_key>' -XPUT "<artifactory_url>/artifactory/<deb_repo>/<group_path>/<package_name>;deb.distribution=<distribution>;deb.component=<component>;deb.architecture=<architecture>" -T <local_file_path>"

In order to use this deb repo from artifactory 

• Add the repo key to apt key

curl <artifactory_url>/artifactory/api/gpg/key/public | sudo apt-key add -

• Add the deb repo into sources.list file (/etc/apt/sources.list)

deb <artifactory_url>/artifactory/<deb_repo> <distribution> <component>

• Then

sudo apt-get update

sudo apt-get install <package_name>

For a generic repo, it should be simpler

• Deploy

curl -H 'X-JFrog-Art-Api:<api_key>' -T <local_file_path> "<artifactory_url>/artifactory/<generic_repo>/<file_path>"

• Resolve

curl -H 'X-JFrog-Art-Api:<api_key>' -O "<artifactory_url>/artifactory/<generic_repo>/<file_path>"

For docker repo, after setting the docker client as from this post. You can use docker pull, docker push with docker command.

For distribution repo, you can refer to the REST API doc

From Jenkins pipeline script, you can use httpRequest plugin for calling the REST API via http request as below:

def artifactoryAPI = '<artifactory_url>/artifactory/api/distribute'

def reqHeader = [

    name: 'X-JFrog-Art-Api', 

    value: "<api_key>"

]

def reqBody = groovy.json.JsonOutput.toJson([

    targetRepo: "<distribution_repo>",

    packagesRepoPaths: ["<deb_repo>/<group_path>/<package_name>"]

])

httpRequest contentType: 'APPLICATION_JSON', customHeaders: [reqHeader], httpMode: 'POST', requestBody: reqBody, url: artifactoryAPI

Using REST API is a raw task so if you want to send all sufficient metadata together with your artifact, you will have to send a lot options with the api. My suggestion of using Artifactory plugin on Jenkins so when you upload an artifact to artifactory, this plugin will help you to add all the metadata also the build info from Jenkins into artifactory. You can open Builds tab then select a specific build, in Build Info JSON tab you will have every information from Jenkins build there (also the Git hash of each scm check out repo under "vcs" object).

How to use Artifactory Plugin

• Configuration: go to Manage Jenkins/Configure System/Artifactory. Enter the proper information, be sure of using the access account with deployment permission
• Usage: Sample below for uploading a debian package

def artServer = Artifactory.server '<artifactory_id>'

def uploadSpec = """{

    "files": [{

        "pattern": "<file_path>", 

        "target": "<repo>/<path>", 

        "props": "deb.distribution=xenial;deb.component=main;deb.architecture=amd64"

    }]

}"""

def buildInfo = Artifactory.newBuildInfo()

buildInfo.env.capture = true

artServer.upload spec: uploadSpec, buildInfo: buildInfo

artServer.publishBuildInfo buildInfo

You can refer the guideline for more actions on artifactory with this plugin, this is a very handy one.

Distribute to Bintray

JFrog provide a stable solution to distribute your package to the internet with Bintray. They have a very simple and good integration model. After few clicks to configure the Bintray integration with Artifactory, you will need to spend more time to study how to setup the forwarding rules in the distribution repo. I myself, can't apply the regex capture rule in the sample so I make very simple rule, such as dedian
Leave the repository filters empty
Bintray output
Repository: a debian repository I created on Bintray
The rest items I use the available token from debian type

A note for generic type, you have to choose a proper layout in order to have the available tokens for the Bintray output. From my perspective, artifactory has applied a very solid structure of Maven package so you can use "maven-2-default" type. I don't know why, if you specify the Bintray Path with the "orgPath" when you deploy a generic package to artifactory then Bintray can't create sub-directory under generic repo, here is my experiment

Deploy a generic package: <orgPath>/<module>/<module>-<baseRev>.<ext>
Setup distribution rule: create a generic package type with "maven-2-default" layout, then Repository is the created generic repo on Bintray, Package is <module>, Version is <baseRev> and especially, Path is <module>.<ext>, then you will have a generic package on Bintray with this structure under generic repo: <module>/<baseRev>/<module>.<ext>

After integrated to artifactory, the task come to simple now, you can call the REST API for distribution the packages from your local repo to the distribution repo then it will sync into your Bintray. Refer to the Groovy sample above for Bintray distribution via REST API. Again, when come back with artifactory API (or CLI) then the Jenkins plugin is quite handy here.

Bintray Entitlement

How about using artifacts on Bitray from external. Bintray also provides the access key and entitlement to manage and control accesses for external usage. With the same mechanism with artifactory, Bintray has all the REST API that offer all the based functions.

Get existing access keys:
curl -X GET -u<user>:<key_api> "https://api.bintray.com/orgs/<org>/access_keys

Get detail of an access key:
curl -X GET -u<user>:<key_api> "https://api.bintray.com/orgs/<org>/access_keys/<key_id>

Create new access key (no expire and can access UI)
curl -H "Content-Type: application/json" -X POST -u<user>:<key_api> "https://api.bintray.com/orgs/<org>/access_keys" -d '{"id":"<new_user>","expiry":-1,"api_only":false}'

This call will return a json with detail user credential and information.

Delete an access key:
curl -X DELETE -u<user>:<key_api> "https://api.bintray.com/orgs/<org>/access_keys/<key_id>

Get existing entitlement:
curl -X GET -u<user>:<key_api> "https://api.bintray.com/repos/<org>/<repo>/entitlements

Create an entitlement on a specific repo for an access key:
curl -H "Content-Type: application/json" -X POST -u<user>:<key_api> "https://api.bintray.com/repos/<org>/<repo>/entitlements" -d '{"access": "r","access_keys":["<key_id>"]}'

So now, you can control the access permission for an external account on specific scope on Bintray (you can take a look on the API detail for specify different entitlement scopes such as product, repo, package or package version)

Same as artifactory, if you want to use the debian repo from Bintray, you will have to:

• Add Bintray public key into your apt key

curl https://bintray.com/user/downloadSubjectPublicKey?username=bintray | sudo apt-key add -

• Add Bintray source repo into your sources.list

"deb https://<user>:<key_api>@<org>.bintray.com/<repo> <distribution> <component>"

If you don't add the public key from Bintray, a workaround is adding [trusted=yes] into the deb repo in sources.list file

"deb [trusted=yes] https://<user>:<key_api>@<org>.bintray.com/<repo> <distribution> <component>"

Alright, it should be good for now! See you on the next topic!