A DSL for Groovy on top of Apache VFS2
Groovy Java Shell
Clone or download
ysb33r Merge pull request #80 from pbchekin/patch-1
Changed S# to S3 in README.adoc
Latest commit e6b358b Nov 13, 2017

README.adoc

Groovy VFS

A DSL for Groovy to wrap around the Apache Commons VFS libraries.

If you like it, then tweet about it using #groovyvfs as the hashtag.

Build Status Windows Build Status

Groovy Library

Download', link=

@Grapes([
	@Grab( 'org.ysb33r.groovy:groovy-vfs:0.5' ),
	@Grab( 'commons-net:commons-net:3.+' ), // If you want to use ftp
    @Grab( 'commons-httpclient:commons-httpclient:3.1'), // If you want http/https
    @Grab( 'com.jcraft:jsch:0.1.48' ) // If you want sftp
])
import org.ysb33r.groovy.dsl.vfs.VFS

def vfs = new VFS()

// Simple copy operation
vfs.cp 'ftp://foo.example/myfile', 'sftp://bar.example/yourfile'

// Utilising the DSL
vfs {

    // Copy file from one site to anther using two different protocols
    cp 'http://first.example/myfile', 'sftp://second.example/yourfile'

    // Not implemented yet - move file between two sites using different protocols
    mv 'sftp://second.example/yourfile', 'ftp://third.example/theirfile'

    // Lists all files on a remote site
    ls ('http://first.example') {
        println it.name
    }

    // Streams the output
    cat ('http://first.example/myfile') { strm->
        println strm.text
    }

    // Create a new folder on a remote site
    mkdir 'sftp://second.example/my/new/folder'

    // Change default options via property Map
    options 'vfs.ftp.passiveMode' : true

    // Change default options DSL style
    options {
        ftp {
            passiveMode true
        }
    }

    // Use options on a per URL basis
    cp 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', 'sftp://second.example/yourfile?vfs.sftp.compression=zlib'

    // Download a compressed archive and unpack to local directory
    cp 'tbz2:ftp:/first.example/myFiles.tar.bz2", new File( '../unpack-here' ), recursive:true

    // Replace content of a file with text
    overwrite 'ftp://first.example/myfile?vfs.ftp.passiveMode=1' with 'this text'
    overwrite 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', { strm -> strm << 'this text' }

    // Append content to a file
    append 'ftp://first.example/myfile?vfs.ftp.passiveMode=1' with 'this text'
    append 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', { strm -> strm << 'this text' }
}

Gradle plugin

Download

From initiation a VFS object has been available as an extension to the project class. The interface is very experimental and may change without much warning in future releases of this plugin.

buildscript {
    repositories {
        jcenter()
        mavenCentral()
      }
      dependencies {
        classpath 'org.ysb33r.gradle:vfs-gradle-plugin:0.5.1'
        classpath 'commons-net:commons-net:3.+'  // If you want to use ftp
        classpath 'commons-httpclient:commons-httpclient:3.1' // If you want http/https
        classpath 'com.jcraft:jsch:0.1.48'  // If you want sftp
      }
}
apply plugin : 'org.ysb33r.vfs'

// Create a VFS task
task copyReadme << {
  vfs {
    cp 'https://raw.github.com/ysb33r/groovy-vfs/master/README.md', new File("${buildDir}/tmp/README.md")
  }
}

// it is also possible to update global options for vfs
vfs {
  options {
    http {
      maxTotalConnections 4
    }
  }
}

If you want to see what VFS is going run gradle with --debug

In 1.0 a copy task has been added.

import org.ysb33r.gradle.vfs.tasks.VfsCopy

task download ( type : VfsCopy ) {

    from 'http://somewhere.example/file' (1)

    from URL_2

    // Set the destination root URI. Files will be copied into that folder
    into URL_ROOT

    // Set options for all of the copy operations.
    // These override global options, but are only applicable for source and destination URLs within this task
    options {
      ftp {
        passiveMode true
      }
    }
}
  1. Copy the file at this URI <2>

Any local source URIs will get reflected as an input file in the TaskInputs, otherwise it is just an input If the destination URI is local, it will get reflected as TaskOutputs as a file

Adding extra plugins

From v1.0 onwards additional plugins can be loaded via a new extend block. For more details see this gist: https://gist.github.com/ysb33r/9916940

SMB provider

Download', link=

A provider for accessing SMB shares is now avavilable. The plugin must be loaded separately.

@Grab( 'org.ysb33r.groovy:groovy-vfs-smb-provider:1.0-beta1' ),
@Grab( 'jcifs:jcifs:1.3.17' ),

vfs {
  extend {
    provider className: 'org.ysb33r.groovy.vfsplugin.smb.SmbFileProvider', schemes: ['smb','cifs']
  }

  cp 'smb://someserver/share/dir/file', new File('localfile.txt)
}

NOTE: when embedding windows credentials in the URL use %5C in place of backslash i.e.

  smb://DOMAIN%5cUSERNAME:PASSWORD@HOSTNAME/SHARE/PATH
Note
This provider has a concurrency bug (#73).

S3 provider (EXPERIMENTAL)

Download', link=

A provider for accessing S3 shares is now available and will be fully supported in future version. The plugin must be loaded separately.

@Grab( 'org.ysb33r.groovy:groovy-vfs-cloud-core:0.1-beta1' ),
@Grab( 'org.apache.jclouds:jclouds-all:1.7.2' )
@Grab( 'org.apache.jclouds.driver:jclouds-jsch:1.7.2' )
@Grab( 'org.apache.jclouds.provider:aws-s3:1.7.2'
vfs {
  extend {
    provider className: 'org.ysb33r.groovy.vfsplugin.cloud.s3.S3FileProvider', schemes: ['s3']
  }

  cp 'smb://id:key@bucket/dir/file', new File('localfile.txt)
}

NOTE: Although S3 does not actually support folders, this is simulated through the use of folder names containing / characters.

Command-line utility

Download', link=

A command-line utility mimicking a number of GNU shell utilities is available.

Documentation

Credits

It is seldom that these kind of libraries happen in isolation. It is therefore prudent that I acknowledge the inputs of others in the creation of groovy-vfs