BigData / NoSQL Operations Workshop: How to Scale Dirty and Influence People
Chef is a powerful tool for maintaining and describing the software and configurations that let a machine provide its services.
The more we can decouple, the more we can scale.
They define Chef as a make for machines and as Git for machines. If
the code is a 100 lines range, it is very easy to be maintenable.
Smaller codes help a lot. Chef helps describe a good high level mental
model for your machine. Even if you do not know Ruby, you can read the
recipes for configuring the machine.
Code is inventory, the more lines of code, the harder it will be to move away from them.
A written code is legacy. The bigger the code, the closer from the Web it is. Plenty of legacy constraints.
Product’s job: give me, Engineering says: No way\^W\^W Yes!
It reminds me of long discussions between developers and product owners. The issue is always the same and comes from a lack of dialog and understanding of the consequences of decision. People should have clear KPI to be able to take decisions.
Do not try to solve problems you would like to have.
Or basically the old “If it is not broken, do not fix it.”
The two speakers are introducing Resque
“Resque (pronounced like “rescue”) is a Redis-backed library for creating background jobs, placing those jobs on multiple queues, and processing them later.”
They also used Flume for log aggregation.
Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.
do not write unit tests for High perf sites. Real world will always bites you. Watch matrix and spend time on recovery techs
Typically you can spend a lot of time writing tests that will never fail, but you are loosing time. It is better to be reactive and monitoring metrics and improve your techniques of recovery. How the code is designed to be tolerant for failures. Basically how is your code planned for failures?
Even if the code smells funny inside, but is super effective, then it is beautiful code
The important is to have code which is resistant to performance issues. Software purity doesn’t achieve that much in face of real world.
do not spend time in meetings, write code.
Engineers spending too much time to try solving issues in a meeting room instead of writing code and testing the results live is wasted time. Play with the real world data.