Friday, May 05, 2017

.deliver_later not working in rake tasks in development for a Rails 5 app

I was just working on a rake task in a Rails 5.0.1 application that sends out emails. I was using the default setup of the :async adapter for ActiveJob (development mode). I was also using the letter_opener gem in development so that emails would pop up in my browser rather than be sent.

Everything looked good when I ran the task with each email using .deliver_now.

In order to let the task finish quickly, I switched the rake task so that each email it generates is enqueued with .deliver_later rather than .deliver_now.

I could see the emails getting enqueued by ActiveJob:

[ActiveJob] Enqueued ActionMailer::DeliveryJob ...

but no emails were popping up in letter opener.

It turns out that the problem was that I didn't understand how :async works. Basically, what's going on is that :async is an in memory queue that only lasts as long as the process that creates it lives. With a rake task, the emails get enqueued correctly (in memory), but then the task ends and the memory queue gets cleared out. But that happens before any processing time has been given to the ActiveJob thread.

To test that things are working in development, any of the following will work:
  • Use .deliver_now
  • Add a sleep(5) to the end of your rake task (just for manual testing ... delete that before you commit)
  • Switch to a different ActiveJob adapter that stores the jobs in some durable queue / db and runs as a separate process from your rake task

Thursday, June 02, 2011

Using Capistrano and Rake for DB Maintenance

I know that periodically it's a good idea to run analyze table on tables in your mysql database so they have updated statistics for the query optimizer to use.

Until now, I've generally done it manually by shelling into a machine, connecting to mysql at the command line and running the commands. Not a big deal, or a big time sink, but it seemed ripe for some automation.

#lib/tasks/database.rake
namespace :db do
desc "Runs analyze table on each of the tables in the DB so that queries can be optimized correctly"
task :analyze_tables => :environment do
cmd = build_db_command('mysql')

conn = ActiveRecord::Base.connection
tables = conn.tables.reject {|i| i == 'schema_info'}

db_command = "echo '"
db_command += tables.collect {|table| "analyze table #{table};"}.join(' ')
db_command += "' | " + cmd

puts "Running analyze tables on: #{tables.join(', ')}"
puts %x[#{db_command}]
end
end


the cool thing here is using ActiveRecord::Base.connection.tables to get a list of all the tables in your database. Once I have those, I turn them into a string of commands that I use echo to pipe to mysql.

echo 'analyze table posts; analyze table comments; analyze table users; | mysql -u deploy -psecret blog_production '


since I have other commands that interact with mysql, I created a method
build_db_command
that builds a mysql command using the configuration in your database.yml file.

#lib/tasks/database.rake

def build_db_command(executable)
abc = ActiveRecord::Base.configurations[RAILS_ENV]
case abc['adapter']
when 'mysql'
cmd = [executable]
cmd << "--host='#{abc['host']}'" unless abc['host'].blank?
cmd << "--user='#{abc['username'].blank? ? 'root' : abc['username']}'"
cmd << "--password='#{abc['password']}'" unless abc['password'].blank?
cmd << abc['database']

cmd.flatten.join(' ')
else
raise "Task not supported by '#{abc['adapter']}'."
end
end


In order to make this rake task accessible through capistrano, I used rubaidh's approach:
#config/recipies.rb
def rubaidh_run_rake(*tasks)
rake = fetch(:rake, '/usr/local/bin/rake')
rails_env = fetch(:rails_env, 'production')

tasks.each do |task|
run "cd #{latest_release}; #{rake} RAILS_ENV=#{rails_env} #{task}"
end
end

namespace :db do
desc "Run through all the tables in the DB and run analyze_table on them"
task :analyze_tables, :roles => :db, :only => { :primary => true } do
rubaidh_run_rake "db:analyze_tables"
end
end


voila! Now I can just
cap db:analyze_tables

Labels: , , , ,

Monday, September 20, 2010

Benchmarking Functional vs Data Based Lists in Ruby

At GoGaRuCo 2010 (Golden Gate Ruby Conference), Jim Weirich (@jimweirich) gave a great keynote on Functional and Object Oriented programming in Ruby. In the pres, he threw up some code to implement lists using a simple data structure and alternatively as Procs that relied on closures to keep state.

It seemed like it would take a lot more work for Ruby to build all these Proc objects, but I wasn't actually sure, so I decided to benchmark it. As it turns out, the functional approach is the slowest, but it's closer to a data based approach than I would have expected.

Here is Jim's code for the data structure based approach:
List = Struct.new(:head, :tail)
Cons = ->(h,t) { List.new(h,t) }
Car = ->(list) { list.head }
Cdr = ->(list) { list.tail }


and here is his code for a Proc based approach (using the 1.9 syntax)
f_Cons = ->(h,t) { ->(s) { (s==:h) ? h : t } }
f_Car = ->(list) { list.(:h) }
f_Cdr = ->(list) { list.(:t) }


just out of curiosity, I also wrote a standard Class based approach as well (this uses the idiomatic object.message style rather than the "bare message" style which is more like what you'd see in a Lisp/Scheme based language)
class List2
attr_reader :car, :cdr

def initialize(car, cdr)
@car = car
@cdr = cdr
end

def self.Cons(car, cdr)
List2.new(car, cdr)
end
end


Just to show that all the implementations are working
#Data based
lst = Cons.(1, Cons.(Cons.(2, Cons.(3, nil)), Cons.(4, Cons.(5,nil))))
raise "Problem with Car and Cdr" unless Car.(Cdr.(Cdr.(l))) == 4

#Functional
f_lst = f_Cons.(1, f_Cons.(f_Cons.(2, f_Cons.(3, nil)), f_Cons.(4, f_Cons.(5,nil))))
raise "Problem with f_Car and f_Cdr" unless f_Car.(f_Cdr.(f_Cdr.(f_lst))) == 4

#Class based
c_lst = List2.Cons(1, List2.Cons(List2.Cons(2, List2.Cons(3, nil)), List2.Cons(4, List2.Cons(5,nil))))
raise "Problem with List2.car and List2.cdr" unless c_lst.cdr.cdr.car


and then I benchmarked them over 100,000 list creations and 5,000,000 extractions of the "4" from inside the list:
require 'benchmark'
c = 100_000
n = 5_000_000
#this benchmarks creates and extracting the "4" from (1, (2, 3), 4, 5)
Benchmark.bm do |rpt|
rpt.report("List Create") { c.times do; Cons.(1, Cons.(Cons.(2, Cons.(3, nil)), Cons.(4, Cons.(5,nil)))); end;}
rpt.report("List Extract") { n.times do; Car.(Cdr.(Cdr.(lst))); end;}
rpt.report("Func Create") { c.times do; f_Cons.(1, f_Cons.(f_Cons.(2, f_Cons.(3, nil)), f_Cons.(4, f_Cons.(5,nil)))); end;}
rpt.report("Func Extract") { n.times do; f_Car.(f_Cdr.(f_Cdr.(f_lst))); end;}
rpt.report("Class Create") { c.times do; List2.Cons(1, List2.Cons(List2.Cons(2, List2.Cons(3, nil)), List2.Cons(4, List2.Cons(5,nil)))); end;}
rpt.report("Class Extract") { n.times do; c_lst.cdr.cdr.car; end;}
end


and got this for results (on a 2.8GHz MacBookPro from 2008)
user system total real
List Create 0.270000 0.000000 0.270000 ( 0.263856)
List Extract 3.230000 0.000000 3.230000 ( 3.238410)
Func Create 0.630000 0.080000 0.710000 ( 0.699868)
Func Extract 6.410000 0.010000 6.420000 ( 6.417965)
Class Create 0.210000 0.000000 0.210000 ( 0.217196)
Class Extract 1.100000 0.000000 1.100000 ( 1.090477)


So the Functional approach is definitely slower that the list based (2.6x on create, 2x on extract), but that was a smaller gap than I expected.

I was also surprised that the Class based approach was so much faster than the List based approach (2.9x on extract) since the List implementation is just using Struct.new. It looks like executing a "stubby proc" is quite a bit slower than running an instance method.

For anyone who wants to play with this, all the code is up on github in a gist (http://gist.github.com/588512)

Labels: , , ,

Wednesday, January 20, 2010

Great resources for getting caching working on your rails app

I spent some time today getting caching of static assets up for a project I'm working on. Before picking a direction, I did a bunch of reading to understand different approaches. At the end of it all, the things I found most helpful were:

Stephen Sykes approach to getting Apache to automatically expire files that have the (automatically generated) timestamps that rails generates with image_tag, etc
his blog post

To test if that's working, you can use curl
curl --head http://yoursite.com/images/yourimage.png?1234567890
(as shown in this post that actually describes a different way of doing things that I didn't go with)

Once you've got caching of images from your erb files working, you can use YSlow to see your grade for Expires headers.

If you're still getting a poor grade for expires headers, it's probably because you have CSS that includes references to images.

In order to fix this, you basically want to append timestamps as query strings to the image URL's in your css file. There are two basic ways to do this.

One is to use controllers to generate your stylesheets from .erb files. This allows you to use the rails image_tag helpers. Since the resulting output files will be cached, this is pretty efficient. For details, see this blog post.

The second way (which I went with) is to use the very sweet plugin css-asset-tagger. This is described by it's authors Redline in this blog post. Basically, this works "automagically" to run through your css files after they are deployed on production and modify them by inserting timestamps after each image.

In either case, once the images in the CSS file also contain timestamps in the query strings, the expirations setup at the top will pick them up as well.

Monday, November 30, 2009

Clearing out content in content_for

In rails, multiple calls to content_for append their body's together and yield the concatenation of all the calls. Conceptually, this looks something like:

<% content_for :title do %>
Hello
<% end %>
<% content_for :title do %>
World
<% end %>


producing "Hello World" when you call
<% yield :title %>


In my current project, I was using redbox to create popup windows (see next post for details). In some cases, I wanted to have multiple popup windows on the same page. However, when I created those popups, each partial had a content_for :title and I was ending up with later popups having titles that contained all the words from previous titles.

After a little digging, I decided I needed a version of content_for that reset the value rather than appending to it. I found some old patches for rails that changed how content_for worked, but I thought it would be cleaner to have a new method that set the content_for variable rather than patching the version in rails.

#config/initializers/set_content_for.rb
module ActionView
module Helpers
module CaptureHelper
def set_content_for(name, content = nil, &block)
ivar = "@content_for_#{name}"
instance_variable_set(ivar, nil)
content_for(name, content, &block)
end
end
end
end


by putting this in /config/initializers it's automatically loaded by rails at startup. You use it just like you'd use content_for
<% set_content_for :title do %>
...whatever...
<end %>


but it clears out the current value of the section before setting it to the content of your new block.

Using a modal layout with redbox for popup windows

In my current project, I've been using redbox for popup windows. By default, these come up as a plain white box, so I wanted to style them with a colored area for a title, a region at the bottom for buttons, etc. So I created a layout that has all the styles

#app/views/layouts/_modal.erb
<div class="modal_box">
<div id="title">
<h3>
<%= yield :title %>
<span><%= link_to_close_redbox 'close' %></span>
</h3>
</div>
<div id="modal_flashes">
<%= render :partial => '/layouts/flashes' %>
</div>
<%= yield %>
</div>


then to create a modal window, I just create a partial with the content I want to appear in the modal window

#app/views/wizard/_advice.html.erb
<% content_for :title do %>
How to...
<% end %>
<div class="modal_content content_section">
<p>As the ...</p>
</div>
<div class="actions">
<%= link_to_close_redbox 'Close', :class => "modal_action_link" %>
</div>


and render that partial in a hidden div on my main view

#app/views/wizard/page1.html.erb
<div id="advice" style="display:none;">
<%= render :partial => 'advice', :layout => '/layouts/modal' %>
</div>


that is pulled by by a link_to_redbox call
<%= link_to_redbox 'advice?', 'advice' %>


In this case, I could have gone farther and pushed more of the styling & divs into the modal template. Then I would have had additional content_for's for :body and :actions. I didn't do that because in other places I needed to have the the popups display forms which included content in the body and submit buttons down in the actions. By making everything other than the title the be the content that was yielded, that was easy to do.

Friday, September 25, 2009

Watching the International Space Station

Allison, Jessie and I just got back from sitting on Skyline Boulevard watching the International Space Station shoot across the sky. It was super bright and it's very cool to watch this "star" streak across the sky and know that there are people living and working on it.

If you want to check it out, go to
http://www.heavens-above.com/

and put in your latitude and longitude to see when the ISS will be visible from where you live.

To find your latitude and longitude, pull up your home address on google maps and then paste

javascript:void(prompt('',gApplication.getMap().getCenter()));

into the place you normally enter URL's.

Thursday, July 16, 2009

DRYing Up Views Between Browser and Mobile Versions (across MIME types)

I was working on an app and decided that in order for it to be truly usable on mobile devices (iPhone, Blackberry, etc), it needed to have views that were tailored for small screens.

I started by registering a new MIME type 'mobile' so I could detect it, and respond_to mobile requests appropriately.
#config/initializers/mime_types.rb


Mime::Type.register_alias "text/html", :mobile


However, I quickly ran into a problem. I was duplicating a lot of code between my abc.html.erb views and my new abc.mobile.erb views. The obvious solution was to use partials. However, the *.mobile.erb views were failing to load the partials I already had.

What I realized after a little playing around was that if you have a view with a declared MIME type (new.mobile.erb) and you try to render a partial


<%= render :partial => 'posts' %>

Rails will look for partials with the same MIME type (_posts.mobile.erb), but it will not find partials with other MIME types (_posts.html.erb). However, the solution is to take the MIME type out of the partial (_posts.erb), then the partial will be found when called from either an HTML or a Mobile view.