Check out these related webinars…

Gulp.js – Another Build System for JavaScript applications

In a previous article we examined ‘Grunt.js’ (aka ‘Grunt’) as a system for building JavaScript applications.

While ‘Grunt’ has been around longer, there is another very popular build system calle ‘Gulp.js’, or ‘Gulp’. Gulp takes a different approach to defining the chain of processing tools that are applied to a set of source files to generate output. In many ways, this approach gives a ‘Gulpfile’ that is easier to read and maintain than a ‘Gruntfile’. Also, Gulp gives us an easier way to use other JavaScript tools without actually writing a plugin. In this article, we’ll take a look at Gulp and see how to apply it to building an AngularJS application.

Installing Gulp

Gulp runs under ‘Node.js’, and it’s published to the main ‘npm’ repository. So, install the command line with

npm install -g gulp-cli

Then, we’re going to need some runtime libraries to run it in our project. So, in the target project, install gulp locally with

npm install --save-dev gulp

Using Gulp – the Command Line

Gulp works by defining ‘tasks’. When you call out a task on a command line, Gulp executes that task.

So, if we have a task called ‘build’, then we’ll run it by calling out:

gulp build

Laying Out Our Development Directory

There really isn’t a standard layout for a project directory in the JavaScript world. There are a few ‘scaffolding’ tools out there, like ‘yeoman’, which we used in a previous post. But for this article, we want to take a look at how to setup the build tool itself, so we’re going to roll it from scratch.

We should start by specifying our build requirements. Here’s my usual list… Fair warning here – I’ve been writing software for almost thirty-five years, in every environment from 6502 machine code (hand-entered one byte at a time into a monitor) to the cloud, and I have opinions. Some of those opinions are different from what the fashionable folk in today’s JavaScript world think.

• If it isn’t in source control (i.e. git, svn, hg or whatever), it didn’t happen.
• We need a clear separation between source files and target files. Nothing generated by a machine should go into source code control. I don’t like machines modifying source code.
• I’m OK with a ‘build’ command that prepares everything for the runtime environment.
• I’m OK with manually starting up that runtime environment. I like knowing that the environment is starting up in a clean state.
• I distrust anything that happens ‘automatically’. Well, most things.
• The ability to use an interactive debugger is not that important to me. I prefer logging and thinking.
• Anything I need to do as a developer should be achievable from the command line in the project folder – i.e. the build tool should be a command-line tool like ‘Gulp’. Following this rule makes it far easier to automate things in a Continuous Integration or Continuous Delivery system.
• HTML is a presentation format, not a development format. In practice, this means that we’re going to need templating of some kind. For this small example, the templating inherent in Angular will be enough. In some cases, we might need more (e.g. processing through Markdown or Handlebars templates, or both).
• CSS3 is also a presentation format, not a development format. So we’ll need a CSS pre-processor. I’m partial to ‘less’, but many people like ‘sass’ as well.
• Unit testing is a good thing. I’ll use the Karma framework here.

I’m also going to assume for this project that I’m just building the browser-side components. So I won’t bother with any kind of server-side JavaScript or So, let’s start with a simple project folder structure that looks like this:

.
├── src
│ ├── main
│ │ ├── js
│ │ ├── less
│ │ └── web
│ └── specs
└── target

We’ll sort out what’s under the ‘target’ folder as we go along. We’ll also see about including the various libraries that we’ll need.

Creating the Gulpfile

The instructions for gulp are contained in a file called ‘gulpfile.js’ resides in the project folder. Gulp will execute this file to configure itself and setup the various gulp tasks.

Let’s create a couple of simple gulp tasks that create a folder called ‘target/web’ and copy the contents of the ‘src/main/web’ folder over to it. Then we’ll create a quick server that serves the contents of ‘target/web’ for testing.

Why do this? Why not just serve the contents right from the source folder? The answer is that ‘source is source’ and shouldn’t be touched by machines. However, the actual resources that we deploy are going to include (eventually) different callouts for scripts, stylesheets, etc, because we’re going to do some processing on them. It may be counter-intuitive, but the JavaScript and css that we actually serve out to web app clients is not source code. It’s compiled code, even though we may not be changing the language (i.e. we’re not compiling from ‘C’ to assembler). We still need to process the contents.

The gulpfile defines ‘tasks’ that do things by making calls to the ‘Gulp API’.

First, let’s put something into ‘src/main/web’ so there’s something to copy. Create a file called ‘index.html’ inside ‘src/main/web’ with the following content:

<!DOCTYPE html>

<html>
 <head>

 </head>
 <body>
 Hi there!
 </body>
</html>

Now, create a file called ‘gulpfile.js’ in the main folder, and put in the following:

var gulp=require('gulp');

gulp.task('web', function() {
 gulp.src('src/main/web/*')
 .pipe(gulp.dest('target/web'));
});

Then, from the command line, you can run:

gulp web

… and you should see that ‘gulp’ copied over all the contents of ‘src/main/web’ to ‘target/web’.

.
├── gulpfile.js
├── src
│ ├── main
│ │ ├── js
│ │ ├── less
│ │ └── web
│ │ └── index.html
│ └── specs
└── target
 └── web
 └── index.html

Let’s break down what we put into the gulpfile. At the top, we said ‘var gulp=require(‘gulp’)’. This is standard JavaScript for ‘Node.js’ that says ‘get me the gulp module and store a reference to it in the variable called gulp’

Then, we call the ‘task’ function on gulp to define a task called ‘web’. The task is contained in the lambda function that follows. Here’s where we start to see the ‘pipelined’ nature of gulp.

This nature builds on Node’s idea of ‘Streams’, which builds on Node’s idea of ‘Event Sources’. It’s worthwhile to read over the Node documentation on both of these to make sure you understand the concepts.

Basically, ‘gulp.src(folder)’ creates a ‘Stream’ that emits ‘File’ objects, one per actual file in the folder that we provide to ‘gulp.src(folder)’. Then, we use the ‘pipe’ function to connect this stream to another stream that processes the file objects, outputting them for use by another stream. Finally, we pipe that output to an stream created by ‘gulp.dest(folder)’. This final stream takes the file objects (they’re an internal representation of the file – we’ll look at them in a minute) and writes them to the physical file system, in the folder that we specify. As a side effect, it creates whatever folders and sub-folders are necessary to do the write.

What’s a File Object?

The gulp documentation casually mentions that the stream carries ‘Vinyl Files’. Vinyl is a sub-project of gulp. It defines a JavaScript class that describes a file. The neat thing about Vinyl files is that they describe (and allow access to) both the path (i.e. where the file is) and the contents (i.e. what’s in the file) of a file. That’s more important when you start trying to extend gulp. For our purposes, we just need to concentrate on the fact that a file in gulp includes both the path and the contents.

The subtle implication here is that a processing step in a gulp pipeline has a lot of possibilities:

• It can simply modify the contents of a file, but leave the name unchanged
• It can modify the contents and filename of a file (e.g. transform a ‘.md’ file into a ‘.html’ file by applying a template)
• It can receive one file and emit multiple files with arbitrary names
• It can receive multiple files and emit a single file with an arbitrary name.

Gulp Plugins

By now you can probably figure out that a gulp plugin is written as a stream processor that takes a stream of ‘File’ objects and outputs a stream of ‘File’ objects. A plugin might act as a filter, making small changes to each file in the stream, or it could act as a file replacer, taking in one or more files and replacing them in the output stream with one or more completely different files.

Some of the most popular plugins you’ll come across:

• gulp-uglify processes JavaScript files into their minimized form, shortening variable names and removing whitespace. Use this to minimize the size of your script files, speeding up the page load. Also known as ‘minifying’ files.
• gulp-concat combines all the files piped into it into one file piped out. Use this on your script files to minimize the number of network interactions the browser needs to make.
• gulp-minify-css does the same sort of minification on CSS files that gulp-uglifydoes on JavaScript.
• gulp-browser-sync sets up a web server to serve out your files for testing. But there’s a special surprise: it inserts a small script that will automatically reload pages when the files change. So you can simply do a build, and then see the browser update automatically. This function saves you having to reload the files by hand, which is neat because it makes it easy to have more than one browser open to the same page. That speeds up testing a lot.

Actually, we’ve cheated a little bit. There is no actual plugin gulp-browser-sync. Because gulp is able to simply run a JavaScript function as a task, we don’t need an actual plugin – we can just use the browser-sync library directly from the gulpfile. We’ll see what that means a little later.

You’ll find a lot of gulp plugins out there (because they’re pretty easy to write, and really useful). Just enter ‘popular gulp plugins’ into your favorite search engine.

Using External Tools – An Example with browser-sync

Basically to use a gulp plugin or an external tool, we need to:

• Install the plugin as a ‘development’ dependency
• Reference the plugin in ‘gulpfile.js’. Usually, we’ll put the plugin into a pipeline, but we can also use it as a standalone task (e.g. for gulp-browser-sync).

Let’s try that out, with browser-sync. On the command-line, enter

npm install --save-dev browser-sync

When that’s done, add the following to the ‘gulpfile’. This adds a gulp ‘task’ that starts the browser-sync server:

var browserSync = require('browser-sync').create();

// Static server
gulp.task('browser-sync', function() {
 browserSync.init({
 server: {
 baseDir: "./target/web"
 }
 });
});

… and then run it from the command line.

gulp browser-sync

You will probably see that a web browser window pops up with our home page, which didn’t have much to it. Notice that the url is ‘localhost:3000’. That’s the url that browser-sync serves out files from.

Leaving that page open, open a different browser and bring up http://localhost:3000

For instance if the first page opened with Firefox, open up Chrome and go to the home page http://localhost:3000

Now you have two browsers open to the same page.

Open another tab on either browser and enter the url [http://localhost:3001]. This is browser-sync’s user interface. Notice the ‘Reload All’ button at the top.

Edit ‘index.html’ so that it reads something else besides ‘Hi there!’. Then at the command line, enter:

gulp web

That builds our system by copying the source page over to our target directory. Now go to the browser-sync UI and press that ‘Reload All’ button.

Voila! All your browsers should have reloaded. That’s pretty cool, except that we still have that manual task of clicking the reload button. No problem, we can add a different gulp task that builds our system and then reloads the browsers.

Add the following into ‘gulpfile.js’:

var http=require('http');
// Build and then tell browser-sync to reload.
gulp.task('reload', ['web'], function() {
 http.get('http://localhost:3000/__browser_sync__?method=reload');
});

We’ve added a gulp task called ‘reload’, that has a list of pre-requisite tasks. In this case the only pre-requisite is ‘web’, which ensures that Gulp runs the ‘web’ task before running the ‘reload’ task. The ‘reload’ task simply issues an http ‘get’ request using Node’s ‘http’ module, to the url [http://localhost:3000/__browser_sync__?method=reload]. That’s a not-quite-RESTful API call to browser-sync that causes it to reload all the connected browsers just as though you clicked the reload button.

So now, edit your ‘index.html’ to something else and then on the command line, enter:

gulp reload

… and all your browsers should reload themselves.

Conclusion

We’ve had a look at ‘Gulp.js’ and seen how it works by connecting ‘plugins’ into ‘pipelines’ that are run by ‘tasks’. We created a simple project to demonstrate it, and we had a look at ‘browser-sync’, which isn’t actually a Gulp plugin, but is easy to use from the ‘gulpfile’. This is considered one of the great advantages of Gulp; we are able to use a lot of functionality from other libraries and tools without actually needing a purpose-built plugin.

That’s it for now. In a later article, we’ll expand our build setup to include CSS pre- and post-processing, as well as templating and JavaScript processing.

Related Webinars

Related Training

Introduction to Angular 2 Programmin

Comprehensive AngularJS Programming