Opportunity Knocking: SWF ISO QA Cowboy

Note: This post is only reflective of my desire to work with someone who doesn't suck, and not of my employer. I have to say, though, that I have considerable pull so get at me if you want to rock with yours truly every day to make some kick ass software that gets used by real people at a well funded start-up.

Here I am, lonely, and looking for a mate... my one and only work mate! I know that you are out there, and you can roll with the punches like the best of them. I know that you are un-phased by "emergencies" and have corrupted your own database a time or two only to breathe and recover gracefully (at least the second time). I know you aren't afraid of restarting a service live if timed right and that you truly understand that logs are there for more than just wasting disk cycles. You know when to yell at me for my coding sins and when to give me slack for a 95% pass day.

Save me from myself and my kludgey disgressions. Are you out there?

MISSION:

Help us get our test on by setting up testing protocols, enforcing test suites, and anything else that makes a sturdier code base and happier customers. We have regression/unit testing in place but are looking for someone to enforce that they get run often as well as wrangle together the rest of the test picture pieces including pre and post deploy testing, bugs, new features, and performance.

A strong head and leadership qualities are a must: we are growing fast so don't be surprised if you are expected to voice opinions and yell about the right way to do things now and then. You will be given any tools you need to maintain system integrity and enforce test policy.This is the perfect position for someone who is ready to step it up a notch and help lead a small company to success-ville.

For those that are interested and able, this position can easily and quickly turn into a developer position.

SYSTEM:

We work primarily with Python and the Plone framework but dabble in many arenas such as COM, XMLRPC, XMLDB, Adobe Livecycle Workflow and Designer, etc... depending on what needs to be glued at the time. You will learn and understand how all these pieces fit together into one beautiful system and are expected to care for each piece like it was written by you.

REQUIREMENTS:

• BS in Computer Science or must possess portfolio proving proficiency otherwise
• 1-2 years experience testing or building web applications, preferably with dynamic languages and kudos if that language is Python
• Ability and excitement to learn new languages/technologies in a web based environment
• Recent grads will be considered if internship experience is close - we will gladly feed and grow you into the position
• Must know when to ask questions and when to ask Google
• Self-sufficient and flexible
• Excellent troubleshooting, analytical, documentation and communication skills
• Must be dependable, have a positive attitude, and be a team player

Knowledge of any of the following is a awesome:

• Experience working with and/or developing in Plone
• Familiarity with Test Driven Development and Agile Methodologies
• History working with XML databases
• Experience with Adobe PDF and/or Livecycle Designer

COMPENSATION:

Competitive salary based on experience with all the startup stock yummies.

I Poop on Designing for Scalability

Dear John -

Is it OK that I answer your question with a blog post? It seems very web 2.0, and my whole goal in life is to be more web 2.0 than any other web 2.0 weenie out there. Plus, maybe there is a reader out there, and maybe they have a better response.

Unfortunately, I've done a bunch of scaling in my short time on this planet (especially working with the notoriously performance agnst Plone) and I have to side with the camp that you should rarely spend time scaling architecture in the beginning. Here are some yummy reasons on why you should wait to pop your scaling cherry:

• your web app may never take off and you have wasted time scaling when you could have spent time writing a killer feature
• if it does take off, pay someone else to worry about it. If you read anything about scaling twitter, read about how they "fired" Blaine Cook and hired a bunch of scalability experts instead. Zing!
  
• caching goes a loooong way. I like squid myself - it's a sexy beast if a little hard to configure - and I hear varnish is pretty top notch too. Hell, httpd has a nice accelerator built in too if you need a quick fix. Let's take a moment and remember WHY caching works so well: it serves up content that an app server like RoR or Django could give two craps about such as javascript, css and images. Have you ever looked at how much time your browser spends loading this stuff? It's a lot, and cache servers know when to expire, refresh, reload http headers, etc. Your cache server may be your users' browsers' best friend.
  
• you can and should throw more hardware at the problem first. its cheap enough these days and it will buy you the time you need to develop a real scaling solution
  
• Jared Spool gave a great talk at sxsw about actual performance and perceived performance. the performance of your pages (i.e. load time) is almost always next to perceived performance, the time it takes your user to complete a task. For example, amazon.com takes on average an unthinkable time to load pages but it is commonly perceived as the fastest site out there because of its 1 click functionality to get things done.
  
• IP latency is a factor. If you are integrating with any external site (who isn't these days?), chances are that you will see more lag time from waiting for responses for that than any kludgey code you can write
• it will rarely clog where you think it will clog so take that pipe dream to the dump and let the bums sleep in it
  

That being said, here are some things you can think about now if you are worried:

• don't write stupid code. if you notice yourself writing 4 nested loops, think - hey, that could be nasty later on.
  
• write good db queries and for the love of god don't use your code to filter out the results. in the same respect, don't access database variables more than needed - make a higher level variable and reference that forever. watch out for this in ORMs - they can be very inefficient for scaling even though they make code writing fast. but read bullet 1 above first.
  
• know your language/tools. if you are using RoR or something that is known to be slow, anticipate it being a problem later and rewriting key parts in a language like C. oh, your language doesn't have C wrappers? that could be a problem ...
• think about concurrency while you are writing for KNOWN expensive operations. Think: does this action rely on another action before going to the next step? For example, we have a process which needs to create an appountment in exchange and then attach 2 files to the resulting appointment. Instead of writing a functional piece of code like so: create appointment (2s) > attach file 1 (60s) > attach file 2(120 s) > report results(2s)) totaling 184sec, think of doing a threaded version: (create appt(2s) > report results(2s) > kickoff concurrent attach threads(120s) ) totaling 124 seconds. Web peeps don't think about concurrency enough. Be different.
  
• put a round robin queue that diverts from dead parents so you can hot patch stuff live if your code requires restart for changes to take effect (i.e. compiled code). this has saved me a hundred times over and eliminates downtime while still allowing you to respond to heinous bugs.
  

Last but not least, don't forget Knuths famous words of wisdom: "Premature optimization is the root of all evil!!!"

This data streams crap seems like it was written by an architecture astronaut. Yeah right! Who really writes code like that? (hint: no one).

Hth,

Liz

>>>

...
So, although I truly want to know how you're doing, I also have a question for ya. It stems from thinking long and hard how to go about building the architecture for my latest startup. I've read interesting posts such as Two data streams for a happy website (http://gojko.net/2008/03/03/two-data-streams-for-a-happy-website/) and Scalability (http://romeda.org/blog/2008/05/scalability.html) from Twitter's Blaine Cook, but it's hard to figure out where to go from here.

Basically... how much time do I spend planning how the architecture can scale before I know the metrics. I've read that I shouldn't worry about it until I'm there, while others say to keep 2 separate data streams (one that requires users to be logged in and one that doesn't). I expect to have to tweak the solution if we do reach limits, but the more I know before I start writing code the better.

Have you (or friends you know) hit this problem? What advice could you provide?

Thanks in advance!

Best,
John

>>>