bundle update gets scary when a single command carries too much weight.

A CVE fix, a rubocop bump, a sidekiq minor release, and a rails upgrade do not belong in the same conversation. But they often land in the same lockfile diff anyway. I do not trust that kind of PR, and I do not expect anyone else to trust it either.

I do not think the problem is Bundler. The problem is treating dependency maintenance like one big chore instead of a few smaller jobs.

Here is the split I prefer.

Nothing clever. Just separate work with separate risk.

Pull security fixes out of the pile

If a gem is vulnerable, handle that first.

gem install bundler-audit
bundle audit --update

Then update only the gem you actually need to move:

bundle update --conservative --patch <gem-name>

I like doing this in isolation because the goal is obvious. You are not “cleaning up dependencies.” You are removing a known problem.

That distinction matters in review and when something goes wrong. A security fix should not sit around waiting for someone to debug a broken formatter or a noisy minor update in a completely different part of the tree.

Move dev and test tooling on its own

After that, I usually take the lower-risk bundle next:

bundle update --conservative --group=development
bundle update --conservative --group=test

# Or:
bundle update --conservative --group test development

If your Gemfile groups things differently, adjust the commands to match. The point is to keep tooling updates away from runtime updates.

This is often the easiest place to make progress. If something breaks here, it usually breaks in CI, in local scripts, or in editor tooling. That is still annoying, but it is a very different kind of risk from changing gems that sit in the request path or talk to production services.

I also like that it gives you a smaller PR with a clear story: we refreshed the tools around the app, not the app itself.

Do runtime updates without the big gamble

This is where people reach for one big bundle update and hope for the best.

I would not.

Start with the cheapest pass:

bundle update --patch --strict

That usually clears a decent amount of drift without rewriting the whole dependency graph.

After that, slow down a bit. Run the test suite. Check the areas that matter. If the app has a few dependencies that tend to cause trouble, read their changelogs before you keep going.

Then use bundle outdated and decide what the next small batch should be. Maybe that is one background job stack. Maybe one API client. Maybe one gem that has been behind for a while and deserves its own PR.

bundle update --minor --strict <gem-name>
bundle update --major <gem-name>

There is no prize for updating everything in one shot. I would rather review a few boring dependency pull requests than stare at one giant lockfile diff and guess.

Treat Gemfile constraints as real upgrade work

At some point, Bundler stops because Gemfile says no.

This is where I switch mental models:

bundle outdated --groups

If a gem is blocked by a version pin or an upper bound, you are no longer doing routine maintenance. You are changing an explicit decision in the application.

That deserves more attention.

Sometimes the change is easy. Sometimes it needs code changes, rollout notes, or a dedicated ticket. Either way, I would not hide that work inside a general dependency refresh.

I care even more about this with gems close to the heart of the app: rails, database adapters, job processing, auth, anything that can quietly change behavior under load.

What this changes

The main benefit is not elegance. It is control.

Each diff has one reason to exist. That makes review less tiring, and when something breaks, you usually know where to look first.

That is the part people actually want from dependency maintenance. Not a perfectly current lockfile. Just a process that does not feel reckless.

Quick reference

• Fix vulnerable gems first.

• Update test and development gems separately.

• Use a patch pass for runtime dependencies before picking off smaller batches.

• Treat Gemfile constraint changes and major upgrades as deliberate work, not lockfile churn.

Problem: calculate and return the current step of a user under a very complex onboarding process, which depends on a myriad of conditions. The implementation should be:

• Easy to introduce new steps in-between existing ones;

• Easy to remove existing steps;

• Easy to maintain, iterating over the current behavior of any existing steps;

The idea

class CurrentOnboardingStepCalculator
  def initialize(user)
    @user = user
  end
  
  def self.call(user)
    new(user).call
  end
  
  private
  
  attr_reader :user
    
  def call
    steps.lazy.filter_map { |step| step.call(user) }.first
  end
  
  # Keep the steps in the order you want the process to happen.
  def steps
    [
      MissingRequirementStep,
      MissingIdentityStep,
      MissingAddressStep,
      MissingBusinessStructureStep,
      MissingPaymentInformationStep,
      MissingTermsAcceptanceStep,
      # And the list goes on.
    ]
  end
end

Pretty simple, right? Not a single if statement nor a case switch. We are telling this class to return the first non-nil value from the array. The lazy.filter_map chains the operation to be lazy-evaluated.

In other words, consider a hypothetical execution:
- MissingRequirementStep returns nil;
- MissingIdentityStep returns nil;
- MissingAddressStep returns :missing_region;

The execution will stop there and return the symbol without realizing the following steps. Then, an API could return this symbol to the application frontend, which is prepared to handle the situation.

Interesting! Now we are free to implement the rules that determine each step we need, keeping them well encapsulated using separate classes for each one.

Let's see how we could implement the MissingAddressStep class to calculate and return the possible values which belong to this step.

class MissingAddressStep
  delegate :address, :subscription, :suspect_fraud?, to: :user, private: true
  delegate :city, :region, :country, to: :address, private: true
  delegate :expired_trial?, to: :subscription, private: true
  
  def initialize(user)
    @user = user
  end
  
  def self.call(user)
    new(user).call
  end
  
  private
  
  attr_reader :user
    
  def call
    return unless request_address?
    
    [
      missing_city,
      missing_region,
      missing_country
    ].lazy.filter_map(&:call).first
  end
  
  def request_address?
    [
      expired_trial?,
      suspect_fraud?
    ].any?
  end
    
  def missing_city
    -> { :missing_city if city.blank? }
  end
  
  def missing_region
    -> { :missing_region if region.blank? }
  end
  
  def missing_country
    -> { :missing_country if country.blank? }
  end
end

Again the lazy.filter_map pattern is applied, mixed with a few lambda objects responsible for checking in each address field of our theoretical example.

To spice things up, I'm illustrating how we could delay this specific onboarding step by early returning if all preconditions are met. Following the same idea, we could apply an exclusive logic to defer or early request a given field within the lambda implementation.

💆 Ah! Please, don't focus on how I named things here. This is just an example. I'm not trying to convey a naming pattern but instead an idea to solve a problem. Given your context, you will likely apply different names, structures, and so forth. Jorge Manrubia put out a nice write-up called "Vanilla Rails is plenty". You should check it out if you are using Rails and want some inspiration to name and organize things within your project.

See you around! 👋

In my previous post, we saw how to use GitHub Actions to deploy your application manually. Now let's learn how to keep a record of your production deployments.

Manual application deployment using GitHub Actions

As you may already know, you can define dependent jobs within your GitHub Actions workflow. So, now let's define a release phase for our deploy job, which will run only when a production deployment is triggered.

name: Deploy

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy'
        required: true
        default: 'staging'

jobs:
  deploy:
    name: Deploy to ${{ github.event.inputs.environment }}
    runs-on: ubuntu-latest
    env:
      echo "Ommited for brevity (see: https://tomasmuller.dev/manual-application-deployment-using-github-actions/)."

    steps:
      echo "Ommited for brevity (see: https://tomasmuller.dev/manual-application-deployment-using-github-actions/)."
      
  release:
    needs: deploy
    if: github.event.inputs.environment == 'production'
    runs-on: ubuntu-latest
    name: Release
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Get last commit message
        run: |
          LAST_COMMIT_MESSAGE=$(git log -1 --pretty=format:"%s")
          echo "LAST_COMMIT_MESSAGE=$LAST_COMMIT_MESSAGE" >> $GITHUB_ENV
      - name: Create New Relic deployment marker
        run: |
          curl -X POST "https://api.newrelic.com/v2/applications/${{ secrets.NEW_RELIC_APP_ID }}/deployments.json" \
            -H "X-Api-Key:${{ secrets.NEW_RELIC_API_KEY }}" \
            -i \
            -H 'Content-Type: application/json' \
            -d \
            '{
              "deployment": {
                "revision": "${{ github.sha }}",
                "description": "${{ env.LAST_COMMIT_MESSAGE }}",
                "user": "${{ github.actor }}"
              }
            }'

Use GitHub Actions secrets to store your NEW_RELIC_APP_ID and NEW_RELIC_API_KEY.

Deploying an app can be an exciting event, but often your app breaks because of bad changes. A record of your application deployments can guide you to the root cause.

New Relic allows you to track deployments to correlate any deployment to your app's performance and anomalies. Also, tracking deployments create deployment markers that appear in APM charts.

After the deployment is recorded using the REST API, you can optionally notify a webhook endpoint of the deployment.

The destination of the webhook can be your Slack instance. To use webhooks to set up a deployment notification for a Slack channel:

1. Log in to your Slack account as an admin, then go to App directory > Manage > Apps.

2. Search for your New Relic app, then select Add configuration.

3. From Post to channel, select an existing Slack channel or add a new channel, then Add configuration.

4. From the list of options, copy the webhook URL.

5. Go to one.newrelic.com > (account dropdown) > Account settings > Integrations > Deploy notifications > Webhook.

6. Paste the Slack webhook URL, then save.

7. Optional: Send a test message.

That's all. Simple and effective. Now, it's your time! Go ahead and ship something!

Did you know that you can configure your workflows to run only when manually triggered through the Actions tab on GitHub, GitHub CLI, or the REST API?

An everyday use case for this feature is when you have multiple environments running your application and wants to release a branch to a specific environment.

Here is a basic recipe for that purpose, making use of workflow_dispatch and event inputs.

name: Deploy

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy'
        required: true
        default: 'staging'
        
jobs:
  deploy:
    name: Deploy to ${{ github.event.inputs.environment }}
    runs-on: ubuntu-latest
    env:
      FLY_APP_CONFIG: ${{ github.workspace }}/${{ fromJSON('{"staging":"fly.staging.toml","production":"fly.production.toml"}')[github.event.inputs.environment] }}
      FLY_APP_NAME: ${{ fromJSON('{"staging":"myapp-staging","production":"myapp-production"}')[github.event.inputs.environment] }}

    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Install Fly.io CLI
        uses: superfly/flyctl-actions/setup-flyctl@master

      - name: Run Fly Doctor
        run: flyctl doctor

      - name: Deploy the application to Fly.io
        run: |
          echo "Deploying to $FLY_APP_NAME with config $FLY_APP_CONFIG"
          flyctl deploy --remote-only

Here I'm giving you an illustration of what could be a very basic workflow to deploy an app to Fly.io.

Also, as food for thought, I'm showing how you could grab different configuration files or values (env section) using the environment input. Then, you can elaborate on the deploy command, applying these values to build the command line and ship your software to the right environment using the correct configuration.

Since it is possible to trigger workflows like this using REST API, you can go above and beyond, connecting with your existing tools and processes.

The following is a recording of my detailed walkthrough of Clojure's macros system on Clojure Brazil Meetup. Slides.

Back in 2013, AWS announced a new service called OpsWorks. Let's look at its actual state and how you can deploy your Ruby on Rails applications there.

OpsWorks is an application management service for managing applications of any scale or complexity on the AWS cloud. You can think of it as something between Heroku or Beanstalk and a manually configured server environment.

• AWS Elastic Beanstalk is an easy-to-use solution for building web apps and web services with popular application containers such as Java, PHP, Python, Ruby, and .NET. You upload your code, and Elastic Beanstalk automatically does the rest. Elastic Beanstalk supports the most common web architectures, application containers, and frameworks.

• AWS OpsWorks is a robust end-to-end solution that gives you an easy way to manage applications of nearly any scale and complexity without sacrificing control. You model, customize, and automate the entire application throughout its lifecycle. OpsWorks provides integrated experiences for IT administrators and ops-minded developers who want a high degree of productivity and control over operations

• AWS CloudFormation is a building block service that enables customers to provision and manage almost any AWS resource via a domain-specific language. You define JSON templates and use them to provision and manage AWS resources, operating systems, and application code.

The main concepts of OpsWorks are:

• Stacks: a stack represents a collection of EC2 instances and related AWS resources that have a common purpose and that you want to manage collectively. Within a stack, you use layers to define the configuration of your instances and use apps to specify the code you want to deploy.

• Layers: a layer is a blueprint for a set of EC2 instances. It defines the instance's settings, resources, installed packages, profiles, and security groups. OpsWorks provides a set of layers for standard use cases such as application servers, which you can customize as needed. You can also create a custom layer whose configuration you can define from the ground up.

• Applications: an application represents code stored in a repository that you want to install on application server instances. When you deploy the application, OpsWorks downloads the code from the repository to the specified server instances.

• Instances: an instance represents a server. It can belong to one or more layers, that define the instance's settings, resources, installed packages, profiles and security groups. When you start the instance, OpsWorks uses the associated layer's blueprint to create and configure a corresponding EC2 instance.

There's no charge for using OpsWorks. You pay only for AWS resources being used like EC2 instances, RDS databases, etc.

OpsWorks uses Chef to configure the layer's EC2 instances. Chef turns infrastructure into code. With Chef, you can automate how you build, deploy, and manage your infrastructure. Your infrastructure becomes as versionable, testable, and repeatable as application code. A Chef recipe defines everything that is required to configure part of a system. And a Chef Cookbook is a collection of recipes.

You don't have to be a MasterChef (oh, 🤦🏻‍) to use AWS OpsWorks. There is a good set of predefined layer blueprints for standard use cases, all of them are open-source, available on opsworks-cookbooks Github repository.

This post focuses on how you can deploy Ruby on Rails applications on AWS OpsWorks. There are fantastic resources on the Internet explaining OpsWorks philosophy and how it works under the hood, so I won't repeat them here. You're invited to check the curated references at the end of this post.

Here is a collection of tips and tricks to help you deliver Ruby on Rails applications using AWS OpsWorks. Enjoy!

Log in to your AWS account. Select OpsWorks on AWS console. It can be found under the Deployment & Management product family.

Tip Nº1: you can have many stacks as you want. Doing an analogy with Heroku, you can think of a stack as a new Heroku application. A completely new and isolated environment. Talking about environments, a common practice is to have multiple stacks that represent different environments, like staging, production, and so on.

Tip Nº2: when you are creating a stack, you have the option to set a Default SSH key for all future instances within the stack. So, a common practice is to create or import a key pair before creating a new stack. To do that, first, go to EC2 dashboard, click on Key Pairs option, and create or import your key pair. By doing so, all layers instances will have it configured and you'll be able to ssh those instances.

Tip Nº3: you can customize layer cookbooks using the custom JSON textarea. Chef cookbooks accept parameters in the form of node attributes. You need to check what parameters you can send to each cookbook (use the source). Check below one useful example:

{
  "opsworks_initial_setup": {
    "swapfile_size_mb": "512"
  },
  "unicorn": {
    "worker_processes": "2",
    "version": "4.9.0"
  },
  "nginx": {
    "worker_processes": "1"
  },
  "deploy": {
    "application_short_name": {
      "database": {
        "adapter": "postgresql",
        "type":"postrgesql",
        "encoding": "unicode",
        "min_messages": "warning"
      }
    }
  }
}

Tip Nº4: the swapfile_size_mb parameter above is especially useful when deploying Rails applications on EC2 micro instances (free tier) and use a deploy hook (that I'll share in minutes with you) to precompile your assets (imitating Heroku deploys). The assets:precompile command eats a lot of memory, causing the instance to run out of memory on every deployment if you don't increase the swap file size.

Tip Nº5: during the stack configuration you have the option to use custom Chef cookbooks. This is useful when setting node attributes aren't sufficient. By setting custom cookbooks you can go further and override some files written out by your layer's recipes, or even build a completely new and customized layer.

Every stack contains one or more layers, each of which represents a stack component, such as a load balancer, application servers, application databases, workers, and so on.

I can't do a good analogy this time with Heroku. However, you can think of a layer as a custom add-on, or your Heroku Postgres (which is an add-on in fact).

Tip Nº1: currently AWS OpsWorks provides a set of layers for standard use cases such as application servers, databases, etc., which you can customize as needed. Also, there is a special kind of layer to connect your RDS databases to your stack. This is perfect if you are coming from Heroku. First, go to RDS dashboard, create a new RDS Postgres database and then go to OpsWorks, select your stack, add a layer, and select the RDS layer pointing to the recently created database.

Tip Nº2: a typical Rails Application will have three or more layers:

• A Rails App Server layer

• A Load Balancer layer

• A Database layer

• N worker layers (yes, like your Heroku workers)

Tip Nº3: when you create the Rails App Server layer you have the option to select the Ruby version and other common settings. If you intend to use a load balancer for your App Server instances (which is highly recommended!), first go to EC2 dashboard and create a Load Balancer before creating the Rails App Server layer. A common practice is to name the load balancer with the stack-name plus the -lb suffix, resulting in your-app-short-name-lb.

Tip Nº4: create and attach a security group for your load balancer configuring the inbound and outbound traffic as follows:

Inbound:

Outbound:

Tip Nº5: if you use HTTPS for your application, you must upload the SSL certificate files to AWS IAM. An SSL Certificate allows you to configure the HTTPS/SSL listeners of your load balancer. You may select a previously uploaded certificate, or define a new SSL Certificate. To upload your certificate to AWS, ensure that all your certificate files are in PEM format. Follow the steps below if your certificate was issued by Comodo:

Extract the certificate files to a folder and cd to that folder:

cd /path/to/certificates/folder
mkdir out

Convert all certificates and private key:

openssl x509 -in ./AddTrustExternalCARoot.crt -outform pem -out ./out/AddTrustExternalCARoot.pem
openssl x509 -in ./COMODORSAAddTrustCA.crt -outform pem -out ./out/COMODORSAAddTrustCA.pem
openssl x509 -in ./COMODORSADomainValidationSecureServerCA.crt -outform pem -out ./out/COMODORSADomainValidationSecureServerCA.pem
openssl x509 -in ./your_issued_certificate_name.crt -outform pem -out ./out/your_issued_certificate_name.pem
openssl rsa -in ./server.key -outform pem -out ./out/server.key.pem

Create the CAChain:

cat ./out/COMODORSADomainValidationSecureServerCA.pem > ./out/CAChain.pem
cat ./out/COMODORSAAddTrustCA.pem >> ./out/CAChain.pem
cat ./out/AddTrustExternalCARoot.pem >> ./out/CAChain.pem

Finally, upload it to AWS IAM:

cd out
aws iam upload-server-certificate --server-certificate-name YourIssuedCertificateName --certificate-body file://your_issued_certificate_name.pem --private-key file://server.key.pem --certificate-chain file://CAChain.pem

Tip Nº6: if you use HTTPS for your application, configure the load balancer health check to use HTTPS for the ping protocol. Set only an / as the ping path.

Tip Nº7: to use an RDS Postgres database layer along with your Rails application, you need to configure and apply a Security Group to your database instance allowing any source IP to access your database. To do that go to EC2 dashboard, select Security Groups option and create a new security group with the following inbound and outbound rules:

Inbound:

Outbound:

Tip Nº8: if you already have a database on Heroku Postgres, and want to migrate all existent data to your RDS Postgres instance, do the following:

Put your Heroku app in maintenance mode:

heroku maintenance:on --app your_heroku_application

Capture and download a backup via command line, or using Heroku Postgres interface.

Uncompress it to raw SQL:

# the `-n public` option restore only objects that are in the public schema
pg_restore -n public -O latest.dump > heroku_latest.sql

And then import into your RDS instance:

psql -f heroku_latest.sql --host=your.rds.database.endpoint --port=5432 --username=your-rds-db-username --password --dbname=your_rds_dbname

Tip Nº9: after creating the Rails App Server layer, you can edit the layer and add additional operational system packages that will be available on all instances. Again, doing an analogy with Heroku you can think of this as "something like" Heroku Buildpacks. I want to reinforce the double quotes here. There's no such thing as Buildpacks on OpsWorks. This is just an analogy. You probably wrote one Buildpack if you already needed to run a package that is not part of the default set of installed packages.

For instance, if you use the pg gem, add the postgresql93-devel and postgresql93-libs packages. Also, don't forget to add nodejs package. We will use it to precompile application assets.

We're almost there. At this point, we have configured the software that will run on the server. Now it's time to create the instances themselves which are the servers that will execute and run the application code.

Tip Nº1: instances can be started manually, or configured to start and stop on a schedule or in response to load patterns.

Tip Nº2: a common practice is to spread your instances over subnets in multiple AZs for higher redundancy.

Adding an app to the stack is the first step to deploying an application to your application servers. An AWS OpsWorks application represents code that you want to run on an application server. The code itself resides in a repository such as an Amazon S3 archive or a GitHub repository.

With the AWS OpsWorks interface is very easy to register an application. Just click on Apps and on the +add link. From now on just fill out the form which is pretty straightforward. Bellow are a couple more tips and reminders.

Tip Nº1: when you deploy an application, AWS OpsWorks triggers a deploy event, which runs each layer's Deploy recipes. Each of the built-in application server layers includes a set of Deploy recipes, which use the data from the stack configuration and deployment JSON to automatically deploy the app's code from the repository to the layer's instances. The Deploy recipes also handle related tasks such as restarting services and setting up database connections.

Tip Nº2: you can use a private git repository as application source. You just have to configure the repository ssh key on both sides (in the OpsWorks application configuration and your repository).

Tip Nº3: if you have multiple environments (staging, production, etc.) you can configure to source the application from a specific branch corresponding to the environment name.

Tip Nº4: select RDS as data source type if you are using an RDS database layer.

Tip Nº5: not exactly a tip, but a reminder: you can (and should!) use environment variables to store your secret keys and third-party credentials. A common practice is to set at least the RACK_ENV and RAILS_ENV env vars.

Tip Nº6: add your application domains or subdomains filling in the Domains section. Use the DNS manager of your domain to configure the records. You have the following options:

• add an A record pointing it to the specific instance IP address. This is not good because this IP changes every time you stop and start the instance.

• grab the instance Public DNS and add a CNAME record pointing to it. Again this is not good, because this address also changes after a full stop/start since the IP address composes the name of instance Public DNS endpoint.

• the correct approach is to use a Load Balancer layer and use the DNS name of the Elastic Load Balancer associated with this layer. This endpoint never changes and all requests will be balanced to all available instances.

Tip Nº7: if your application uses HTTPS you have to configure the SSL certificates. You just have to enable the SSL switch and fill the textareas with the contents of your purchased certificate files as follows:

• SSL certificate: enter the SSL certificate (usually: -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE-----). Comodo issued certificates can do the following:

cat your_issued_certificate_name.crt COMODORSAAddTrustCA.crt COMODORSADomainValidationSecureServerCA.crt AddTrustExternalCARoot.crt > bundle.crt
# copy the bundle contents and paste into the SSL certificate textarea
pbcopy < bundle.crt

• SSL certificate key: copy and paste the contents of the key generated with your certificate (normally this file is called server.key).

The SSL certificates of Certification Authorities textarea is optional, used for intermediate CA key or Client Authentication.

Tip Nº1: write your own deploy scripts and use the aws command line tool to automate the application deployment of your OpsWorks applications. The following command creates the deployment of an AWS OpsWorks application:

aws --region='<stack-region>' opsworks create-deployment --stack-id='<stack-opsworks-id>' --app-id='<app-opsworks-id>' --command='{\"Name\":\"deploy\"}'

Just replace the <stack-region>, <stack-opsworks-id> and <app-opsworks-id> with the corresponding values for your stack and application. These ID's you find viewing the stack and the app respectively.

Tip Nº2: you probably configured a default SSH Key for your instances (tip number 2, under adding stacks). Having a configured SSH key allows you to SSH directly an instance by its IP address or Public DNS. As one example, if you want to manually check the Rails application logs on a running instance:

# Attention: the default username can change depending on instance's operational system.
# Open the Instances list under OpsWorks and check how to connect to a specific instance clicking on its ssh link.
ssh -i ~/.ssh/your_ssh_key ec2-user@instance.ip.address
sudo su deploy
cd /srv/www/your_app_short_name/current/log
tail -f staging.log

Tip Nº3: Chef deploy callbacks can help you to automate your deployment scripts. Just create a directory in your app called deploy and add files named for the appropriate callbacks. For example, here's how you can handle assets pre-compilation, expose environment variables during the deployment, and perform common database tasks like running migrations and seeds.

deploy/before_restart.rb

rails_env = new_resource.environment["RAILS_ENV"]

Chef::Log.info("Mapping the environment_variables node for RAILS_ENV=#{rails_env}...")
node[:deploy].each do |application, deploy|
  deploy[:environment_variables].each do |key, value|
    ENV[key] = value
  end
end

Chef::Log.info("Precompiling assets for RAILS_ENV=#{rails_env}...")
execute "rake assets:precompile" do
  cwd release_path
  command "bundle exec rake assets:precompile"
  environment "RAILS_ENV" => rails_env
end

deploy/after_restart.rb

rails_env = new_resource.environment["RAILS_ENV"]

Chef::Log.info("Migrating database for RAILS_ENV=#{rails_env}...")
execute "rake db:migrate" do
  cwd release_path
  command "bundle exec rake db:migrate"
  environment "RAILS_ENV" => rails_env
end

Chef::Log.info("Seeding database for RAILS_ENV=#{rails_env}...")
execute "rake db:seed" do
  cwd release_path
  command "bundle exec rake db:seed"
  environment "RAILS_ENV" => rails_env
end

I encourage you to play with AWS OpsWorks. With a very simple interface, almost any developer can model and map the components of their applications. Plus you can take advantage and use other services available on AWS Cloud. For example, CloudWatch to monitor your applications and resources, creating custom alarms, or using hundreds of metrics and graphs. Also, take a look at RDS database monitoring options — you will be amazed. Also, there are many other services available. I can't list all of them here.

And don't get me wrong: I'm not comparing OpsWorks with Heroku in any way. For me, they are entirely different things. Seriously, I already tested a lot of services out there, and I continue checking every time something new appears on the scene. However, I was not able to find a competitor to beat Heroku's ease of use. All analogies in this post are to help "Heroku-only" developers to understand OpsWorks vocabulary and how it works.

The simplicity of OpsWorks makes it a reliable option for delivering your applications. Highly productive, powerful, and flexible, supporting any software that you would like to use with scripted installations.

If you are looking for another choice to deliver your applications, definitely it's time to give AWS OpsWorks a chance.

Eleven. This is the right answer. At least for now where the latest Ruby version is 2.1.5 this number remains the same. Check the source for yourself.

So what?

Now and then I keep hearing Ruby programmers complaining about the memory usage increase when switching from Ruby 2.0 to Ruby 2.1. The objective of this blog post is to help those people who are stuck on MRI 2.0, giving an overview of each one of the GC tuning parameters, and explaining how you can benefit your app by using them.

Sorry, this kind of thing does not exist when the subject is memory profiling or performance tuning. There's no silver bullet or one answer for all. Just profile, experiment, analyze and repeat. A lot.

"Everything in Ruby is an object" - you probably already heard this statement. So, a running Ruby program is made up of objects. Thousands of them. Thanks to the Garbage Collector you don't have to manage memory allocation manually.

Each major release of Ruby has included improvements to the garbage collector, most of them about speed. This is because for the GC to do its work it needs to pause the program execution while it is running.

Since the birth of Ruby its garbage collector is based on the Mark & Sweep algorithm, which is made up of two phases (Effective Ruby, 2014):

1. Traverse the object graph (the Ruby heap). Objects that are still reachable from your code are considered alive and subsequently marked.

2. Next, any objects that were not marked in the first phase are considered garbage and swept away, releasing memory back to Ruby and possibly to the operating system.

Now, can you imagine how expensive is to traverse the object graph of a running program with thousands and thousands of objects? That's why the team behind this complex piece of software engineering concentrates so much effort on seeking quality and performance improvements.

Here's the history of CRuby's GC evolution (Incremental GC for Ruby interpreter, K.Sasada, 2014):

• 1993/12 Ruby 0.9: Conservative mark and sweep GC
   - Simple algorithm
   - Easy to implement C extensions

• 2011/10 Ruby 1.9.3: Lazy sweep To reduce pause time on sweep

• 2013/02 Ruby 2.0: Bitmap marking

• 2013/12 Ruby 2.1: RGenGC

According to with K.Sasada work, we can expect another major update on Ruby 2.2 GC which among other things, will introduce incremental GC algorithm into major GC to reduce long pause time (RincGC: Restricted Incremental GC algorithms).

The garbage collector in Ruby 2.1 introduced the Generational GC, using Mark & Sweep algorithm to maintain the Ruby heap classifying objects into two categories (generations).

This technique treats new objects differently than older ones. A new (young) object is one that has just been created by your program. An old object (also referenced as a mature object) is one that will continue to be used by your program.

The reason behind this categorization is based on the weak generational hypothesis, which means: "most objects die young". In other words, new objects will probably die young. Having this "life expectancy" differences between new and old objects, different GC algorithms and strategies can be applied to each generation.

Enter the minor and major GC runs:

• Minor GC: should be faster than major GC, run often, traversing only the young objects.

• Major GC: traverses the whole object graph, including the old generation.

To classify an object as young or old the GC does the following:

1. On a minor GC, when it marks an object in the mark phase, it promotes the object to the old generation (remember: when an object is marked, means that the object will survive the current GC run).

2. Sweep away unmarked objects.

3. During the next minor GC, the entire old generation section will be ignored by the GC, traversing only the young generation.

Above is just the basics for you to understand the GC tuning parameters topic below. The entire garbage collection process implemented in Ruby 2.1 (RGenGC) is much more complex in order to solve all existing problems. For a detailed view of how it works take a look at this post's references.

When Ruby 2.2 goes out we can expect another major update to the garbage collector, which will probably introduce a three-generation GC.

Before we continue let's see some key points about the Ruby heap:

• The heap is the memory managed by Ruby and to which we have access.

• The heap is divided into pages and slots.

• Each heap page holds 408 slots (GC::INTERNAL_CONSTANTS[:HEAP_OBJ_LIMIT]).

• Internally, a Ruby object is represented as a struct called RVALUE

• Each heap slot has the size of one RVALUE, which is 40 bytes (GC::INTERNAL_CONSTANTS[:RVALUE_SIZE]). So, sufficient size to hold one Ruby object.

• The size of each heap page is around 16kb (408 * 40 = 16320 bytes).

Ok, now we have the minimum amount of information to understand the existent GC tuning environment variables.

Following the idea of each application should be treated as a unique case, there are defaults set on Ruby VM that you have to know and likely want to adjust according to the app weight, pondering with the computing resources at your disposal.

Just to be clear, each one of these GC tuning parameters has a default value, meaning that if you don't set this variable on your application environment, the default value will take place.

So, if your app starts running out of memory after upgrading to Ruby 2.1, this is a good place to start.

In the case of a Rails application, there are a lot of other things that you can do (and probably will have to), most of them related to the application server. Later in this post, I'll give a bonus topic for those who are using Unicorn as an application server.

Roll up your sleeves. Below is the explanation behind each GC tuning variable available today.

1. RUBY_GC_HEAP_INIT_SLOTS: Initial number of slots allocated on Ruby's heap. Increasing this value from its default can reduce GC activity during application boot. Default value: 10000

2. RUBY_GC_HEAP_FREE_SLOTS: After a GC execution, the minimum number of free slots that should be available. Default value: 4096

3. RUBY_GC_HEAP_GROWTH_FACTOR (new from 2.1): When the heap needs to be expanded, allocate slots by this factor: (next slots number) = (current slots number) * (this factor). Default value: 1.8

4. RUBY_GC_HEAP_GROWTH_MAX_SLOTS (new from 2.1): Set the maximum number of slots that Ruby is allowed to add to the heap at once. When disabled, Ruby uses the heap growth factor to determine by how much to grow the heap. Default value: 0 (meaning that this limit is disabled)

5. RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR (new from 2.1.1): Factor to control major GC timing, given by: control threshold = this factor * number of old objects from the last major GC. The control threshold is compared with the current number of old objects. If the control threshold is exceeded by the number of old objects since the last major marking phase, another major GC will happen. Special note: if you want to disable generational garbage collection, you can specify 0.9 (any number lesser than 1.0). Default value: 2.0 (meaning that old objects have to double to trigger another major GC)

6. RUBY_GC_MALLOC_LIMIT: Internally the VM keeps track of malloc_increase, which is the number of bytes that have been allocated but not yet freed. This is effectively the memory growth of the process. This parameter holds the minimum value for GC.stat[:malloc_limit]. If malloc_increase exceeds the malloc_limit a minor GC is triggered. Default value: 16777216 (16MB)

7. RUBY_GC_MALLOC_LIMIT_MAX (new from 2.1): Sets the maximum value for GC.stat[:malloc_limit] from going too high (malloc_limit changes dynamically by the growth factor below, tuning number 8). To remove this upper limit set the value 0. Default value: 33554432 (32MB)

8. RUBY_GC_MALLOC_LIMIT_GROWTH_FACTOR (new from 2.1): The growth factor that controls the GC.stat[:malloc_limit] value grows over time. Given by: new malloc_limit = current malloc_limit * this factor. Default value: 1.4

9. RUBY_GC_OLDMALLOC_LIMIT (new from 2.1): The old generation memory growth is tracked separately by the VM in oldmalloc_increase. This parameter holds the minimum value for GC.stat[:oldmalloc_limit]. If oldmalloc_increase exceeds the oldmalloc_limit a major GC is triggered. Default value: 16777216 (16MB)

10. RUBY_GC_OLDMALLOC_LIMIT_MAX (new from 2.1): Sets the maximum value for GC.stat[:oldmalloc_limit] from going to high (oldmalloc_limit changes dynamically by the growth factor below, tuning number 11). Default value: 134217728 (128MB)

11. RUBY_GC_OLDMALLOC_LIMIT_GROWTH_FACTOR (new from 2.1): The growth factor that controls the GC.stat[:oldmalloc_limit] value grows over time. Given by: new oldmalloc_limit = current oldmalloc_limit * this factor. Default value: 1.2

Obsolete:

• RUBY_FREE_MIN -> RUBY_GC_HEAP_FREE_SLOTS (from 2.1)

• RUBY_HEAP_MIN_SLOTS -> RUBY_GC_HEAP_INIT_SLOTS (from 2.1)

Again: all listed environment variables above are for Ruby 2.1. Ruby 2.2 will probably have an impact on these settings. Tune them only if you really know what you are doing.

"One thing I have learned in a long life: that all our science, measured against reality, is primitive and childlike - and yet it is the most precious thing we have." ― Albert Einstein

Think about. This Einstein's quote fits perfectly here.

In order to accomplish this hard work surround yourself with great tools. A few examples below:

• Ruby GC::stat - Return a hash with interesting information about the garbage collector.

• Ruby ObjectSpace - The ObjectSpace module contains a number of routines that interact with the garbage collection facility and allow you to traverse all living objects with an iterator. The objspace library extends the ObjectSpace module and adds several methods to get internal statistical information about object/memory management.

• NewRelic - The Ruby VMs tab under monitoring menu gives a good set of graphics about the situation of memory and garbage collection.

• Librato

• Skylight

• Heroku App Dashboard Metrics tab

• gc_tracer

• memory_profiler

• Rbkit - Ruby Profiler

• Derailed Benchmarks

Also, the What I Learned About Hunting Memory Leaks in Ruby 2.1 blog post from Peter W. describes an approach to collect and analyze heap dumps from your running application. Heap dumps are the devil's house. Precious raw data lives there.

There's a technique called Out-of-band GC (OOBGC) created to optimize the GC process for web applications. The main idea is to run the GC in-between requests rather than during requests. This technique was first introduced by Unicorn.

Traditionally the OOBGC implementations force a GC run every N requests. This works well however, it does not observe if the GC actually is needed.

Aman Gupta created a more intelligent OOBGC implementation built on new APIs and events offered in Ruby 2.1 and its RGenGC implementation, allowing the VM to make the best decision about when a garbage collection is required.

The installation is pretty straightforward:

1. Add the gctools gem to your Gemfile and bundle install it.

2. Edit the config.ru file adding lines 3 and 4:

# (...)
require 'gctools/oobgc'
use(GC::OOB::UnicornMiddleware)
run Rails.application

Note for Ruby 2.2:

If you are using Ruby 2.2 you don't need OOBGC, since it includes the RIncGC (Incremental GC) and will not pause the world for a long time.

Here we scratched the surface of the most important part of Ruby internals - the Garbage Collection. This invention first appeared around 1959, and since then it continues to be heavily researched, evolving over time through the development of complex algorithms and techniques.

We also saw how the existent GC tuning environment variables can impact your application performance, but more important than blindly trying random values on these parameters is to know your application and the environment where it is running.

That said, in this context, you can't manage what you don't measure and monitor. That's why you have to configure a good set of profiling tools, monitor the memory growth of your application processes, discover the pain points of your application, fix what can be fixed, and repeat.

I hope this rundown can be useful for you.

Graphs are data structures that are highly useful for understanding and represent many real-world problems in all areas, such as business, government, and science.

To take advantage of graph databases we don't need to take a master's in Graph Theory. Instead of that, we must understand what a graph is, and be able to build one by drawing it on paper.

Mathematically speaking, a graph is just a collection of vertices and edges. Or if you don't like math, a set of nodes and relationships that connect them. Graphs represent entities with nodes (vertices), and the way that entities relate to each other is expressed by relationships (edges).

If you stop now and think about it, this structure allows us to model countless scenarios, from commercial systems to more complex problems such as optimization algorithms.

This graph model is formally known as a Property Graph. A property graph has the following characteristics:

• It contains nodes and relationships.

• Nodes contain properties (key-value pairs).

• Relationships are named and directed, and always have a start and end node.

• Relationships can also contain properties (key-value pairs).

Despite being intuitive and easy to understand, the property graph model can be used to describe almost all graph use cases.

As you probably are thinking, graph databases use the graph model to store data as a graph, with a structure consisting of vertices and edges, the two entities used to model any graph. In addition, you can use all the algorithms from the long history of graph theory to solve graph problems in less time than using relational database queries. I'll be covering some of them in my next posts.

Beyond the image above, and now talking specifically about Neo4j, it is an open-source graph database supported by Neo Technology, that stores data using the Property Graph model. It is reliable, with full ACID transactions, expressive, with a powerful, human-readable graph query language called Cypher, and simple, accessible by a convenient REST interface or an object-oriented Java API.

Enough theory and talking for now. Let's prepare our environment to play a little with Neo4j, and build a simple Rails application.

Installing Neo4j on development machines is very easy. If you are on OSX and is using brew, go ahead and issue brew install neo4j on a terminal window.

Or, if you prefer, follow these five steps:

1. Download the Neo4j Community package.

2. Unzip on your installations folder, let's say ~/Applications/.

3. Create a symbolic link named neo4j to the unzipped folder. For instance: ln -s ~/Applications/neo4j-community-2.1.2 ~/Applications/neo4j.

4. Create an environment variable named NEO4J_HOME pointing to this symbolic link.

5. Change the PATH environment variable, adding the NEO4J_HOME/bin to it.

This way, in the future, when you want to update the Neo4j database on your machine, you can just download the new version, unpack, and update the symbolic link pointing it to the new version.

When you have it installed, open a terminal window and type: neo4j start. This command will start the Neo4j server on your machine. Now go check it on your browser accessing http://localhost:7474/. You'll be presented with a super nice administration panel, where you can visualize the data stored on your Neo4j instance, manipulate data using the Cypher Query Language, check all instance configurations, and more.

Neo4j is built on top of Java and the rock-solid JVM. As we want to use (MRI) Ruby on Rails here, let's connect our app using Neo4j's REST API.

To make things simpler, we'll use the awesome gem (surprisingly) called neo4j from @andreasronge. Version 2.x is the stable version. But here we will use it directly from the master branch where version three is under active development, which enables us to use the MRI Ruby connecting to Neo4j via its REST interface. If you are into JRuby, you can even use the stable version and connect using the embedded DB (by filesystem), which means a Neo4j instance running on the same JVM of your app.

Another option to connect an MRI Ruby application to Neo4j using the REST interface is the Neography gem from @maxdemarzi.

This blog post is intended to introduce you to the world of Graph Databases, giving some basic theory about graphs and a practical hands-on using Neo4j and Rails.

For future posts expect to read more about Neo4j, Cypher Query Language, and traversal algorithms.

So, what about learning by doing? And remember: graphs are everywhere!

We had some amazing entries from all over the globe and from a diverse set of languages, and we thank everyone for participating. The overall winner will receive the new iPad, and the language winners receive $100 to ThinkGeek.

The proud new owner of a new iPad, and our overall winner of the Neo4j Heroku Challenge is…

Luanne Misquitta, Flavorwocky

Flavorwocky won the most votes among the Neo4j community and highest ranking on Gensen, recognized for creativity, usability, and all around awesomeness. A big congrats from the whole Neo4j community to you Luanne – we are trying to get that iPad to India as soon as possible!

Language Winners

FlattrWhat – Ruby by Simon Gate
The “Flattr what?” application is a super simple recommendation engine for Flattr users, a micro-payment/tipping system for bloggers and other content creators.

FrostyMug – PHP by Josh Adell
Frosty Mug helps navigate the vast ocean of beers on tap at a proper brew pub. Start with a beer you like, then Frosty Mug will recommend similar beers.

Neoflix: Browser – Javascript by Max de Marzi
In a great spin on the classic Neo4j dataset, Neoflix explores a movie database with D3-powered visualizations to make film recommendations.

Node Neo4j Template – Node.js by Aseem Kishore
The Node template demonstrates how easily Node.js developers can tap into the power of Neo4j for creating a social graph.

Flask Demo App – Python by Carson McDonald
The app is a simple data explorer with bookmarking ability, using user-generated content from Stack Exchange.

Scala Plays with Neo4j – Scala by Andy Petrella
With this through-the-web edited social graph, Andy demonstrates how easy it is to access Neo4j from Scala while running on the Play framework.

GraphTag – Clojure by Aran Elkington
Graphtag is a Clojure application which monitors Twitter mentions against a custom account “GraphTag” and inputs the content of those tweets into Neo4j as a node.

Neo4jOrbust – C# by Romiko Derbynew
Neo4j or Bust is an impressive mashup of Heroku (to access Neo4j) and AppHarbor (to host the .NET application), to discover and follow interesting people.

🏆 NeoQuotes – Java by Tomás Müller
NeoQuotes delivers a stock-exchange symbol lookup application, with integrated voice search!

Lightbulb – Python by James Thornton
Lightbulb is a a Git-powered, Neo4j-backed blog engine for Heroku, using James’ own excellent Bulbs framework for graph databases.

I have to give a shout-out to the Neo4j Community Team and our friends at Heroku for this awesome contest, and for all of the great work put in.

Till next time! 👋

In this post, I will cover all the main details regarding developing my entry for the Neo4j Challenge.

The main objective of this challenge is to create a Heroku-ready template or demo application using Neo4j. So, I thought to myself: — what kind of application would be nice to show up in this contest?

After many ideas, here it is! In short, the application is a Stock Exchange symbol lookup using Neo4j and your voice.

Ok! Let me explain what I did.

My main objective was to try to combine all the joy of Ruby programming, with the power and ease of use provided by the awesome Spring Data Neo4j, Spring Framework, and JVM, to this amazing piece of software called Neo4j which let me curious since 2011 in the Jim Webber talk at QCON conference of that year in São Paulo.

So, to start things up, JRuby comes in as a perfect fit. It also allowed me to use Sinatra to easily develop the user interface (both, voice interface and GUI). I choose Sinatra in this case because of its simplicity and as long the application stays small, it beat Rails (as said by @DHH).

Jetty is another very nice (and old) project, which provides a Web server and javax.servlet container, plus support for Web Sockets, OSGi, JMX, JNDI, JASPI, AJP, and many other integrations, and can be used to develop self-contained applications. In this article from Heroku blog, we have a nice introduction to creating a Java Web Application using Jetty. Its integration with Maven was also another motivation point.

Talking about Maven, possibly in POM.xml resides one of the trickiest parts of the application.

After configuring all the dependencies, aspectj-maven-plugin, jetty-maven-plugin, let's look at the configuration of appassembler-maven-plugin and jruby-rake-plugin:

jruby-rake-plugin:​

<code>
  <executions>
    <execution>
      <id>install-bundler</id>
      <phase>process-resources</phase>
      <goals>
        <goal>jruby</goal>
      </goals>
      <configuration>
        <args>-S gem install bundler --no-ri --no-rdoc --install-dir vendor/bundle</args>
      </configuration>
    </execution>
    <execution>
      <id>bundle-install</id>
      <phase>process-resources</phase>
      <goals>
        <goal>jruby</goal>
      </goals>
      <configuration>
        <args>script/bundle install --without development:test</args>
      </configuration>
    </execution>
  </executions>
</code>

As we can see, the plugin execution is responsible for downloading and installing Bundler, then next executing the specified script which lives in the application script folder. This script basically starts the Bundler client (pre-configuring ENV['GEM_HOME'], ENV['GEM_PATH'] and ENV['BUNDLE_GEMFILE']), and following the parameters, all gems except the ones for development and test groups, are installed.

Last, but not least, the appassembler-maven-plugin:

<code>
  <configuration>
    <assembleDirectory>target</assembleDirectory>
    <extraJvmArguments>-Djruby.native.enabled=false -Djruby.cext.enabled=false -Djruby.compat.version=1.9 -Xmx256m -Xss2048k</extraJvmArguments>
    <programs>
      <program>
        <mainClass>org.jruby.Main</mainClass>
        <name>jruby</name>
      </program>
    </programs>
  </configuration>
</code>

Above we have the configuration for the assemble goal execution. The trick here is to generate the executable with JRuby (org.jruby.Main) as the main class. And you can also set extra JVM arguments, like the version of Ruby language to use.

After this brief look into POM.xml, let's see how Heroku starts Jetty. This is defined in the Procfile:

web: sh target/bin/jruby -S config/jetty.rb

So, that's it! Now our app is running on Heroku Cloud! For more details take a look into /config/jetty.rb to see how Jetty is being started up and /src/main/webapp/WEB-INF/web.xml to see the JRuby-Rack configuration. Other technical details you can find in the application readme file.

Now, let's talk about Neo4j.

Back in 2011, Jim Webber let me very interested in Neo4j with his talk at QCONSP 2011. Maybe because that year I was working on my final graduation project — a recommender system to support fundamental analysis of the Stock Market using natural language processing, and I never forgot one thing that he said. More or less like this: "MapReduce processing between HTTP requests can be very expensive! Some problems cannot be resolved with this approach.". Well, I did this. And I must confess: a graph database would have changed a lot my project.

So, when I discovered this challenge, I decided "- ok, now is the time to take a closer look at Neo4j.".

The first thing I did was access the Neo4j download page and downloaded the 1.6 Community version.

After that, the server installation. It is super easy. Just follow these instructions.

So, what's the next step? Learn, learn and learn!

I was able to make some initial experiments very fast after a dynamic reading of this superb online resource: Good Relationships: The Spring Data Neo4j Guide Book. Thank's to Michael Hunger, David Montag, Andreas Kollegger for this material.

Also, I learned a lot from the following links:

And that's it. This is the magic of open-source software. Curious about how one given thing works? Go to the source code and read it! All you need to get started with Neo4j is this. Trust me ; )

I used Arbor.js library to display the Neo4j data to the user. But how differently the information can be visualized? Through voice interfaces of course!

As I already worked with Voice Interfaces some years ago (in 2008 actually, when I won the Sun Students Contest. More info here and here), this was not a mystery. Again I used VoiceXML markup language, and the free service provided by Voxeo. Refer to the links above for more information about VoiceXML.

Sinatra eased this part even more, through its easiness of building APIs-like applications. All VXML interfaces and grammars are below the /app/views/voice folder and there are only two voice routes in application.rb.

Using the Voxeo service is simple. You only need to create an account, and after that, set up a new application through Voxeo Application Manager, informing the voice entry point of your application (the full URL to your route. In my case to the get /voice/index.vxml route).

Note that your application needs to be accessible by Voxeo server. So don't forget to git push Heroku master, before setting up your app on Voxeo.

Finishing, you'll get the numbers for your application. I always tested with a Skype VoIP number. Works very well here in Brazil and is totally free.

You can try calling to NeoQuotes! This is the Skype VoIP number: +990009369996189207 (to see all available numbers for NeoQuotes access the application and click on the respective top nav bar link).

Important: When you spell the stock symbol, say one letter, and wait for the voice to repeat what you just said. And go on. When you finish, you can say "Okay!", "Go!" or "Search!" for example. If the symbol has an '.' you have to say "dot". The voice will not repeat this (will stay in silence). Just continue with the letters or numbers.

This demo application contains an interesting stack of technologies, providing an environment to develop robust and testable Neo4j applications, with a lot of fun. Certainly much more can be done and improved. And you can help fill issues and submit pull requests!

Please, help to vote in this application at Gensen repository or help with a Tweet!

Thank you Neo4j for running this challenge! It was so enjoyable to participate.

Hohoho! Christmas is coming early this year!

Yesterday the final results for the MySQL and GlassFish Sun Student Reviews Contest were published! Guess who the grand prize winner for the general students' category is? Yes! It's me! Thank you all and congratulations to all participants!

Below is the winners' list:

General students category​

Grand Prize Winner: Tomas Augusto Muller, Universidade de Santa Cruz do Sul, Brazil (review, project).

Second Prize Winners:

• Ronaldo Prass, Unisinos, Brazil (review, project) (shared with Vitorio Sassi)

• Tejesh Morla, University of Illinois at Springfield, USA (review, project)

• Varun Bhatia, University of Delhi, India (review, project)

Sun Java Ambassadors category​

Grand Prize Winner: Kolli Bharath, Dhirubhai Ambani Institute of Information and Communication Technology, India (review, project)

Second Prize Winners:

• Aadhar Mittal, Delhi University, India (review, project)

• Jose Maria Silveira Neto, Universidade Federal do Ceará, Brazil (review, project)

• Qi Cao, Wuhan University, China (review, project)

• Vitorio Sassi, UFRGS, Brazil (review, project) (shared with Ronaldo Prass)

Have you ever heard about VoiceXML? It's a W3C standard that allows you to build web pages that are accessed by voice! It can be used for services like travel tickets selling, hotel reservation, bank account information, and so on. In this post, I'll give you a general view of how to make a basic hotel reservation application with VoiceXML and how to run it using Glassfish and MySQL.

First of all, for VoiceXML applications, we need a voice gateway. I used Voxeo, it's free and provides you with phone numbers that you can call for free using Skype. All you need to do is create a free account in Voxeo and register your application by informing its URL. After registering you'll receive the unique phone number for your application.

I'll assume you already know the basics of VoiceXML syntax for the rest of this post. If you are completely new to VoiceXML, I recommend you check it out first.

The user may use his phone to call the voice gateway. The voice gateway will interact as a voice browser, saying the output of the VoiceXML application and waiting for user inputs. It'll use the internet to access the VoiceXML application, which is deployed in a Glassfish V2 UR2 application server for dynamic content generation. The database used to store user information and hotel reservation is a MySQL 5.1 Community edition.

The Glassfish server receives the voice gateway HTTP requests, queries the MySQL database, and returns a web page with the requested information in VoiceXML syntax. Normally it'll be a form with some input fields that must be filed by the user. The voice gateway then asks for user input and, once it's finished, the information is sent back to the Glassfish server as an HTTP GET or POST.

Another option for the user is to access it right from the web. In my hotel reservation example, only the query for hotel reservations is allowed.

On the Glassfish side, I used the Java Server Pages technology with the Spring framework and iBATIS as the Data Mapper framework. From Spring, we used Spring MVC for the MVC design pattern, Spring ORM for the Spring Dao pattern, and the dependency injection pattern.

This allowed me to easily create the application with all the functionality I needed. Note that the MVC pattern was applied in the voice application and the acronym is still valid. Instead of View, we have Voice. For database modeling, MySQL Workbench was used, a great free tool for this task.

The database ended up looking up like this:

another MySQL Workbench screenshot, showing the VoiceHotel Database Model

Once the database is created, you just need to generate the SQL script from MySQL Workbench and execute it with MySQL. The CLI command to do this is:

mysql -u user -p < script_file.sql

where user is the database user and -p will ask you for the password.

Other options are using your IDE for doing this or MySQL Workbench. The main Java IDEs are NetBeans and Eclipse. Both IDEs allow the connection with MySQL and the automatic deployment in Glassfish. Using Eclipse the deployment can be done by coding a deploy task in one Ant build file or using another mechanism like Maven. This makes application development even faster.

With MySQL Workbench we have a set of nice and useful features like Import Reverse Engineer MySQL create a script for importing a database from script, allowing us to edit this new database, and save these modifications. The databases can be exported using a feature called Forward Engineer SQL CREATE/ALTER script. Another nice thing is the visual insert editor, making it a lot easier pre-populate the database with data.

With the Java and the VoiceXML code ready, we need to put it online. With Glassfish, all you need to do is either, use your IDE to deploy it with one click or put the .war file in the autodeploy folder.

Another possibility is to make use of Glassfish Admin Console. With one Glassfish "fresh installation" this can be done by accessing http://localhost:4848/. The default administrator username is admin and the default password for admin user is adminadmin.

After logging in, all you need to do is access the Web Applications section, click on the deploy button, select your war file and that's it.

VoiceXML applications are a great way to give more than one access possibility to a service and using Glassfish and MySQL for developing this kind of application is incredibly easy. You barely take notice of them :)

You can access the voice hotel application by calling +990009369996079911 from Skype and visualize the data by accessing the backoffice admin panel.

The project is located at http://voicehotel.dev.java.net and you can download all the source code. More information you can find here.