10 recipes for gradle-download-task

I received a couple of requests lately concerning common problems when downloading files in Gradle build scripts using gradle-download-task. Almost all of these issues could be solved by just using gradle-download-task the right way instead of extending the plugin. I have collected a number of example build scripts for these issues over the last months, and I thought it would be a good idea to summarize them here.

For those of you who don’t know what gradle-download-task is: the plugin provides a Download task that displays progress information while downloading files, just like Gradle does when it fetches an artifact from a repository.

The following examples have been tested with Gradle 5.0 or later and gradle-download-task 5.0.0 or later.

Recipe #1: download a single file

This is the simplest use case for gradle-download-task. The following snippet downloads a single file to the project’s build directory.

task downloadFile(type: Download) {
    src 'https://repo.maven.apache.org/maven2/org/citationstyles/styles/1.0/styles-1.0.jar'
    dest buildDir
    overwrite true
}

Alternatively, use the download extension to download a file anywhere in your build script:

task downloadFile {
    doLast {
        download.run {
            src 'https://repo.maven.apache.org/maven2/org/citationstyles/styles/1.0/styles-1.0.jar'
            dest buildDir
            overwrite true
        }
    }
}

Complete example build scripts: simple/build.gradle and simple-extension/build.gradle

Recipe #2: download multiple files

If you want to download multiple files at once to the project’s build directory, use the following snippet.

task downloadFiles(type: Download) {
    src([
        'https://repo.maven.apache.org/maven2/org/citationstyles/styles/1.0/styles-1.0.jar',
        'https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-server/9.1.3.v20140225/jetty-server-9.1.3.v20140225-javadoc.jar'
    ])
    dest buildDir
    overwrite true
}

Complete example build script: multiple-files/build.gradle

Recipe #3: download and rename multiple files

Download multiple files into a target directory and specify a destination file name for each of them as follows:

/**
 * Define files to download and destination file names
 */
ext.urls = [
    'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD': 'config.guess',
    'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD': 'config.sub'
]
 
task downloadFiles(type: Download) {
    src urls.keySet()
    dest buildDir
    eachFile { f ->
        f.name = urls[f.sourceURL.toString()]
    }
}

Complete example build script: multiple-files-rename/build.gradle

Recipe #4: custom HTTP headers

Download a single file and specify a custom HTTP header with the following snippet.

task downloadFile(type: Download) {
    src 'https://repo.maven.apache.org/maven2/org/citationstyles/styles/1.0/styles-1.0.jar'
    dest buildDir
    header 'User-Agent', 'gradle-download-task'
}

Complete example build script: custom-header/build.gradle

Recipe #5: download all files from a directory

The following snippet downloads all files from a directory on a server. It makes use of Ivy’s URL lister to read the server’s directory listing.

task downloadDirectory {
    doLast {
        def dir = 'https://repo.maven.apache.org/maven2/de/undercouch/gradle-download-task/4.1.2/'
        def urlLister = new org.apache.ivy.util.url.ApacheURLLister()
        def files = urlLister.listFiles(new URL(dir))
        download.run {
           src files
           dest buildDir
        }
    }
}

You also need to include a build script dependency to Apache Ivy to make this recipe work. Put the following code right at the beginning of your build script:

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'org.apache.ivy:ivy:2.5.0'
    }
}

Complete example build script: directory/build.gradle

Recipe #6: mirror servers

Download a single file from a mirror server with the following snippet. It first configures multiple mirror servers and then uses the first one that is working.

task downloadFile {
    doLast {
        def mirrors = [
            'http://repo.maven-non-existing.org/maven2/org/citationstyles/styles/1.0/styles-1.0.jar',
            'https://repo.maven.apache.org/maven2/org/citationstyles/styles/1.0/styles-1.0.jar',
            'http://repo.maven-non-existing2.org/maven2/org/citationstyles/styles/1.0/styles-1.0.jar'
        ] as Queue
        while (true) {
            def mirror = mirrors.poll()
            try {
                download.run {
                    src mirror
                    dest buildDir
                    overwrite true
                }
                break
            } catch (Exception e) {
                if (mirrors.isEmpty()) {
                    throw e
                }
                logger.warn("Could not download file. Trying next mirror.")
            }
        }
    }
}

Complete example build script: mirrors/build.gradle

Recipe #7: Lazily evaluate src and dest

The following snippet downloads a single file. It uses Groovy closures to generate values for the src and dest properties at runtime.

/**
 * Example data and methods
 */
ext {
    downloadToBuildDir = true
}
 
def getMavenCentralUrl() {
    return "https://repo.maven.apache.org/maven2/"
}
 
def getStylesJar() {
    return "org/citationstyles/styles/1.0/styles-1.0.jar"
}
 
/**
 * The actual example
 */
task downloadFile(type: Download) {
    src {
        def mavenUrl = getMavenCentralUrl()
        def stylesJar = getStylesJar()
        return mavenUrl + stylesJar
    }
    dest {
        if (downloadToBuildDir) {
            return buildDir
        }
        return File.createTempDir()
    }
    overwrite true
}

Complete example build script: lazy-src-and-dest/build.gradle

Recipe #8: temporary .part file (i.e. download and rename)

This snippet downloads a single file conditionally using a temporary file name (<filename>.part). It renames the file afterwards if the download was successful. The .part file is always overwritten if it exists (e.g. from a previous attempt to download the file).

task downloadFile(type: Download) {
    src 'https://repo.maven.apache.org/maven2/org/citationstyles/styles/1.0/styles-1.0.jar'
    dest new File(buildDir, "styles-1.0.jar")
    overwrite true
    tempAndMove true
}

Complete example build script: temp-rename/build.gradle

Recipe #9: unzip downloaded file

The following two tasks download a ZIP file and extract its contents to the project’s build directory.

task downloadZipFile(type: Download) {
    src 'https://github.com/michel-kraemer/gradle-download-task/archive/1.0.zip'
    dest new File(buildDir, '1.0.zip')
}
 
task downloadAndUnzipFile(dependsOn: downloadZipFile, type: Copy) {
    from zipTree(downloadZipFile.dest)
    into buildDir
}

Complete example build script: unzip/build.gradle

Recipe #10: verify checksum of downloaded file

The final snippet consists of two tasks that first download a file and then verify its checksum.

task downloadFile(type: Download) {
    src 'http://www.example.com/index.html'
    dest buildDir
}
 
task verifyFile(type: Verify, dependsOn: downloadFile) {
    src new File(buildDir, 'index.html')
    algorithm 'MD5'
    checksum '84238dfc8092e5d9c0dac8ef93371a07'
}

Complete example build script: verify/build.gradle

More snippets

At the time of writing this post, the gradle-download-task repository contains 14 example build scripts in both Groovy and Kotlin. You can find them in the examples directory:

https://github.com/michel-kraemer/gradle-download-task/tree/master/examples

More examples will be added in the future, so make sure to check this directory too. If you have other snippets worth adding to the list, send me a pull request on GitHub or post a comment below. Thanks!

More information

If you want to learn more about the gradle-download-task plugin, have a look at its README file or at my earlier post.

If you or your company use any of my projects or like what I’m doing, please consider sponsoring me so I can continue maintaining and developing my projects and new ones!

Thank you so much for your support!

Profile image of Michel Krämer

Sponsor Michel Krämer on GitHub Sponsors

Researcher, software developer, musician, and photographer. I love open source 🥳

Profile image of Michel Krämer

Posted by Michel Krämer
on 3 April 2016

Next post

gradle-download-task 3.0.0

I’m happy to announce a new version of the popular gradle-download-task. The plugin’s core has been completely rewritten. It now supports proxy servers and creates target directories automatically. Updating is recommended for all users.

Previous post

bson4jackson 2.7 has just been released!

I’m happy to announce a new version 2.7 of bson4jackson, the popular library that adds support for binary JSON (BSON) to the Jackson JSON processor. The latest update is a maintenance release.

Related posts

New major version 5.0.0 of gradle-download-task

I’ve just released gradle-download-task 5.0.0. The new version now downloads multiple files in parallel, executes concurrently with other build tasks, and offers better support for Kotlin and Gradle 8.

Build Scala projects with Eclipse Buckminster

Buckminster is a tool to build Eclipse RCP applications. It contains a lightweight Eclipse SDK and features but no means to build Scala projects yet. This post tries to bridge this gap.

Scala projects with Eclipse PDE Build

The Scala IDE does not support Eclipse PDE yet. This article explains how to configure your OSGi-based project, so you can run your PDE build in headless mode and compile your Scala code.