May 20, 2007

Posted by John

Tagged presentation, railsconf, railsmachine, and server

Older: Friday: Evening Keynotes

Newer: Saturday: Memcaching Rails

Saturday: Virtual Clustering

Exploring Virtual Clusters for Rails Development and Deployment
By Bradley Taylor

virtual machine

  • behaves like virtual server
  • operating system
  • many per physical server
  • apartment complex
  • aka: vps, slice, container, accelerator, vm

shared hardware

  • processor (multi-core)
  • memory
  • storage (RAID)
  • network

dedicated resources

  • guaranteed memory, storage and cpu usage

why?

  • consolidate servers
  • isolate applications – helps you avoid shared hosting disaster
  • replicate configurations
  • utilize all available hardware
  • tune resource allocation

isolation

  • protect applications from each other
  • different operating systems
  • different stacks for different apps
  • avoid problems of shared hosting
  • contain intrusion

replication

  • easily create from images
  • standardized deployment
  • automation

utilization

  • fully utilize cpu (mostly idle in web apps, allows apps to share cpu power)
  • reclaim unused memory
  • reclaim unused storage

resource allocation

  • give a server exactly what it needs (smaller memory for svn and such)
  • grow/shrink on demand or with a quick reboot
  • load balance servers on hardware
  • guaranteed vs. burstable (servers can use up to the entire cpu or split if 2 request at the same time)

some numbers

  • 40+ host servers => 400+ virtual machines
  • 4 data center cabinets
  • 256mb to 2gb
  • 10gb-100gb storage
  • 30-40 per 16

no going back

  • powerful
  • flexible
  • low cost (virtualization is the answer to simple low cost solution)
  • rapid
  • security

virtual clusters

  • multiple virtual servers working together
  • shared infrastructure
  • application and services layer
  • servers are almost “free”

many vs. one

  • more than one application, service, customer, version, technology

gains for rails

  • development environments (stages)
  • memory isolation
  • resources to support different traffic
  • protect against php/java (a customer survived the front page of digg with mephisto and a 256mb slice)
  • acessible multi-server scaling

deployment tools

  • subversion – common configuration
  • capistrano – automate updates
  • rake – stack assembly

virtual cluster examples

production/staging

  • two vps
  • released and development version
  • test changes
  • test stack upgrades
  • user acceptance

web-app/db/staging

  • three vps
  • shared database
  • ready to scale
  • resource tuning

mixed languages

  • one vps per technology
  • perfect stack
  • legacy support
  • shared services (db, mail, dns)
  • no “dirty” feeling

high availability

  • two or more physical servers
  • hardware redundancy
  • multiple application pairs
  • easier shared storage (SAN)
  • easier with load balancer

multiple applications

  • one server per application
  • shared services
  • customer isolation
  • rapid deployment

specialized roles

  • each virtual has a job
  • standard: web, app, db
  • expanded: mail, ftp, dns, file
  • virtual data center: load balancer, firewall

application scaling

  • requires foundation
  • more cpu
  • more memory
  • parasite servers

case study: eastmedia

  • development (runs off trunk)
  • staging (runs off branch)
  • production (runs off release tag)
  • 2x 4gb physical → 8 virtual

case study: boom design group

  • low traffic on some (4 apps per server)
  • each high traffic gets own server
  • 2gb physical memory → 9 virtual servers

summary

  • application driven
  • service driven
  • customer driven
  • resource pools
  • infrastructure (share servers ie: db, etc.)
  • less physical servers (so you can be more efficient with resources)

0 Comments

Sorry, comments are closed for this article to ease the burden of pruning spam.

About

Authored by John Nunemaker (Noo-neh-maker), a web developer and programmer who has fallen deeply in love with Ruby. More about John.

Syndication

Feed IconRailsTips Articles - An assortment of howto's and thoughts on Ruby and Rails.

Feed IconRails Quick Tips - Ruby and Rails related links.