THIS IS A DRAFT;)
This is my first blog post in 2 years, so bare with me. Not sure about google sites as a blogging platform though ...
I am a ruby developer and a part time Agile Coach. I have been evangelizing about User Acceptance Testing aka "Specification by Example" with cucumber since 2008. My biggest project involving cucumber happened inside a betterplace.org btw 2009..2010. We used it to communicate with the Product Owners and run acceptance tests (a 30 minutes build without parallel_tests). I aslo wrote courgette, a Rails engine to browse features. If you want to know more, I recommend you follow @gojkoadzic and buy his book "Specifications by example" or even better see him live at a conference: he is a real performer;)
I have felt the pain in 3 different projects - I am about to tear into parts my t-shirt with a giant cuke icon - it's time to move on!
It seems that I am not the only one;)
"Where exactly should I put my step definitions? What if they're corelated - can I abstract them in a simple way? Can I reuse them across projects? Could I even test them?
With cucumber, you usually run into this kind of situations and there's no easy way to get over it. You should use better step file namings perhaps? Create some methods that live next to each other in the cucumber World? Nah, it just doesn't feel good."
Josep Jaume Rey, 21st of October 2011
"In the beginning, it was fantastic, the overhead of Cucumber was gone, we were insanely productive. But over time, cracks appeared. As the projects grew larger, the tests became more and more difficult to maintain."
Jonas Nicklas, 17th of November 2011
My 2 main concerns:
rspec-set or before :all to faster unit tests. There is no equivalent in cuke & co.
Note: I haven't used these libraries in real projects.
Both Spinach and Turnip only run on ruby 1.9.
Both use plain text features a la cucumber, parsing it with Gherkin for Turnip and a gherkin-ruby for Spinach, which unfortunately only supports english at the moment. This is an issue, as an Agile coach living in non-english speaking country (France!), I want to write features in my native tongue.
Turnip has global steps but it is possible to scope them using tags. In Spinach, each feature has its own steps (so no more global steps). That solves the problem of steps madness.
Both have dropped regular expressions! Bravo:) In Turnip, you can however use placeholders in your step definitions, like this:
step "there is a monster called :name" do |name| @monster = Monster.new(name) end
Spinach only supports plain text step but to be honest, that's not a big deal. Scenarios should be written at the right level of abstraction for your customers, in a short and declarative style. Placeholders can encourage your to write scenarios in an imperative style.
I disagree with the design choice of plain text features which need to be parsed in order to be executed as tests. One one hand, as I wrote earlier, it is SLOW. On the the other hand, I can't help thinking it would be so much practical to have it all in 1 place. Having to read multiple step definitions which are not sorted because they are "re-usable" and match them with the scenario is a mental effort which slows me down.
It can render features as plain text in Gherkin format which is cool to communicate with customers. It's easy to DRY step definitions are the DSL is pure ruby. Example:
module LoginSteps def given_I_am_logged_in Given "I am logged in as a Guest" do log_as_guest end end
Something feels weird: it is built on top of Test::Unit which is an Unit Testing framework and was never designed to run acceptance tests. The theory of unit tests is that all test_ methods should be independent and can be run in any order (there is more of course). By contrast, an acceptance scenario is a sequence of steps which must be run in order.
Compare to Spinach top notch logo, website and documentation, coulda feels old school. It is just an impression, its codebase is small and easy to grasp but it does not seem to be an active project. Spinach runs scenarios with its own runner, which makes a lot of sense to me. It is supported by codegram amazing team and the code is lovely ;) Disclaimer: I have met only once with these guys, in euruko 2011 but I don't own them anything!
Turnip is an extension for RSpec. It also makes me feel awkward. On one hand, RSpec is not the fastest test framework, it has too much magic. Check this blog post, http://blog.rawonrails.com/2012/01/very-cursory-test-of-rspec-28-speed.html to see what I mean.
Don't get me wrong, I have been using RSpec since the beginning of 2007 and I am not sure still if I'll try minitest/spec.
On the other hand, look at the code base, it is not all tighted to RSpec and could be perfectly independent of RSpec and used with Test::Unit or minitest. A final note, Jonas and elabs are very active in the testing ruby community (Thank you for capybara;) No doubt turnip will be an active project.
This will take time, probably most of 2012, but I would love if the ruby cuke community works together on this.
I am trying to do my bit with this blog post. I have done more: I have written some code!
As far as I am concerned, the big win is the time saved when writing feature text: you don't have to synch the ./features/my_cool.feature & steps/my_cool.rb files. It's very easy to generate automatically the plain text feature using a Spinach reporter.
A final note, solving Spinach mono linguist short sighting will be easy too. No Gherkin parser is needing, we can tweak the Spinach DSL to add language and write a simple Spinach reporter based on https://github.com/cucumber/gherkin/blob/master/lib/gherkin/i18n.yml
http://ruby-ecosystem.heroku.com/#11 100 slides which give a good introduction
https://www.ruby-toolbox.com/ Impossible to re-invent the wheel
http://rubyonrails.org/ecosystem MVC webapp framework
http://basecamphq.com/ from 37signals SaaS
http://37signals.com/svn/ blog from @dhh who invented Rails
http://infoether.com/ruby-and-rails-whitepaper sounds good
Client heureux de simyo en Espagne et en Allemagne, j'ai tout naturellement souscrit a ce meme operateur en France, pays réputé le plus cher d'europe pour les communications par telephone mobile et détenteur du record mondial des amendes pour entente sur les prix par a mafia SFR Orange Bouygues...
Je n'ai pas digérré le changement de tarif imposé au début 2010. Ma mère et ma compagne qui sont devenus des clients de Simyo m'en veulent. Le fait de perdre son credit non consommé est une héresie qui n'arrive que dans notre merveilleux pays. En Angleterre, je suis client de Virgin mobile depuis 6 ans, je recharge tous les 2 ans et mon numéro n'expirera jamais ni mon credit !
Ces derniers temps, le repondeur de simyo a connu des dysfonctionnements qui ont eu des consequences facheuses dans notre vie. Des messages laissés par l'hopital sur le repondeur n'ont pas été notifiés par SMS et comble de l'horreur, le message a mis pres de 15 jours pour arriver dans le système. Malgré les nombreux appels en absence, la consultation du répondeur aboutit toujours par un pathétique "vous n'avez pas nouveaux messages".
Cher ami google, j'espere que de nombreux internautes trouveront cette page et quitteront simyo ou ne souscriront pas de contrat.
Alors faites comme moi: appelez le 807 depuis votre mobile simyo (appel gratuit) pour obtenir votre code RIO
Je vais tenter l'expérience chez prixtel.com http://www.lowcostmobile.fr/comparatif/prixtel-zero-forfait-mvno-2010 et vous tiendrait au courant. A dire vrai, je n'ai pas beaucoup d'espoir dans la perfection du capitaliste non regulé ...
L'ART semble impuissante face au lobby des 3 gros parasites et j'espere que le salut viendra de l'Europe pour qu'enfin l'on puisse profiter de contrats sans engagement et sans expiration du numéro, sans expiration du crédit et avec un 3G non illimité abordable. Exemple simyo.de : 1Go pour 10 € par mois sans contrat!!!
Une note finale pour les régulateurs: nationalisez l'infrastructure! Quel a été le surcout engendré par le triplement d'installation d'antennes relais???
Imaginerait-on une dupplication du réseau d'egouts, d'eau potable ou du reseau electrique???
Update 30/092010: simyo.fr admet qu'ils ont un probleme. Au lieu de prevenir leur clients et de leur expliquer le probleme (voir http://gettingreal.37signals.com/ch14_Publicize_Your_Screwups.php), ils preferent que ceux-ci se sentent trahis et perdent toute confiance en leur technologie. Pathétique!
=> sauf que les messages ne sont pas disponibles sur le repondeur immediatement
Le déploiement du système informatique du Pôle emploi suspendu
LEMONDE.FR avec AFP | 07.06.10 | 19h06
e déploiement du sytème informatique Neptune, qui fusionne les logiciels de l'ANPE et des Assedics suite à la fusion de ces deux services, "a été suspendu pour éviter un blocage complet", a annoncé la direction du Pôle emploi, à la veille d'un appel à la grève mardi contre les mauvaises conditions de travail.
L'objectif du nouveau système est en test depuis plusieurs mois, mais "il y a des incidents" selon la direction : "dans les quelques régions tests, notamment Aquitaine et Poitou-Charente, les utilisateurs sont susceptibles de buter sur 'une durée anormale de connection au poste de travail' et le 'chargement incomplet du profil utilisateur'".
LE PRÉCÉDENT GÉODE
Ce n'est pas la première fois que les services de l'emploi butent sur des problèmes informatiques. Au milieu des années 1990, l'ANPE avait souhaité développer une application informatique, baptisée Géode, finalement abandonnée en septembre 2005, sans jamais avoir démarré, en dépit de l'importance des dépenses engagées (118,4 millions d'euros à comparer au devis initial de 26 millions d'euros), selon la Cour des comptes.
Les pannes participent à la situation difficile au Pôle emploi, liée à la surcharge de travail et à la mise en place progressive de la fusion, selon l'intersyndicale appelant à la grève mardi formée du SNU-FSU (premier syndicat), de la CGT, SUD, le SNAP, et localement certains élus FO. Ils dénoncent le mal-être croissant des agents, la lenteur des négociations sur le stress et la précarité de certains personnels (le Pôle emploi compte 15 % de CDD et contrats aidés).
A partir de la nouvelle plateforme Waka du gouvernement, hébergée chez Skyrock, le pirate aurait exploité une faille pour accéder à un serveur, sur lequel il a rajouté un fichier nommé « Coucou », ainsi que divers scripts. De cette façon, il aurait potentiellement eu accès à 32 millions de mots de passe d’utilisateur et donc à leurs comptes et leurs blogs.
Zataz.com rappelle également, dans son article, que Skyrock avait déjà été contrôlée il y a plus d’un an par la CNIL qui avait pu constater que les mot de passes, bien qu’étant stockés dans un serveur sécurisé, n’étaient pas cryptés. Un problème qui ne semble pas avoir été corrigé depuis, laissant ainsi tout loisir au hacker de les piocher et de les lire sans autre forme de complication. Bien sûr, les blogueurs peuvent bien modifier leurs mots de passe non cryptés autant qu’ils le souhaitent, mais ce n’est pas ça qui devrait empêcher les hackers d’y mettre la main.
Il suffit d'une facture qui traîne ou utilisée comme justificatif de domicile pour tout savoir sur votre consommation d'électricité, via le site d'EDF. Mieux : la faille permet de gérer votre contrat à votre place, jusqu'à fermer votre compteur.
May 21, 2010
RAIL travellers trying to get away for the bank holiday weekend have been unable to buy tickets this afternoon due to a technical problem.
Au ministère du Budget, on confirme l'objectif annoncé par le précédent ministre, Eric Woerth : le 1er janvier 2011, tous les ministères passeront à Chorus. Un an plus tard que prévu au lancement du projet, et près de dix ans après le vote de la Lolf (loi organique relative aux lois de finances). Chorus est le volet informatique de cette loi.
Les fonctionnaires ne sont pas les seuls à douter. Dans un rapport de 2009, la Cour des comptes estimait que Chorus compliquait l'évaluation des comptes publics. Et en mars, elle était revenue à la charge dans un courrier à Eric Woerth.
Les députés sont encore plus critiques. En juillet 2009, la commission des Finances de l'Assemblée nationale concluait ainsi un rapport sur Chorus :
Des « affirmations non confirmées » ? Le calendrier trop optimiste, mais aussi des coûts sous-estimés et des économies surévaluées. Les coûts ? Le plafond prévu par l'Inspection générale des finances, dans un rapport de 2006, est déjà atteint. Les députés ont revu les coûts à la hausse :
25 projects Bruno Michel AF83
redis: coup de coeur key / value, easy to expire some keys (start_with funk*)
mongodb: Doc DB, learning curve to learn easy to learn from SQL
riak: doc DB, more complex than mongodb, less operator, Map Reduce on documents, content _type bin / json. Easy to set up a distributed archi, not mature, graphs DB
cassandra: for hackers, made in facebook. Very high volumes (twitter) . Dynamo / Bigtable (google DB)
Hard to learn / deploy, riak is easier
neo4J: graphs DB, for some problematics: friends of friends. 1 machine only, no scaling
resque: queuing, made in github,
beanstalkd: is also good, replacing crontab, very simple
puppet / chef : data center automation
chefsolo : for 1 machine
vagrant: I missed
eventmachine: ruby library, fast simple
ErrorNot : af83 Web UI from multiple machines, open source . ruby / php / python
LamsonProject: MVC for email with routes, mailing list management, antispam
Processing.js : graphics on js, ruby binding
Strophe.js : XMPP from browser (eg: presence)
Jasmine: BDD js with plugins for testing node.js
http-console: in node.js command line for http, manage cookies very easily, for stuff hard to test with firebug
hummingbird: node.js analytics on node.js, real time. Nice to look at but no real value
Realie: real time collaborative code editor node.js, hard to install
sass : ruby
less.js : js port of less in ruby in node.js or run directly in browser. Syntax same as standard CSS + variables, easier for designer
ShakaCSS : af83, cross browser js bookmarklet , features which are not available in web developer. Visual
bcat: pipe to browser utility. tail -f mylog.log | bcat some browser, ruby ?
node.js : no package management, hard to set up some stuff
Ryan : for scripting network programs
HTTP lib -> streaming : faster than nginx, thin; Due to V8 VM+ callbacks
* default response: chunked & keep alive
* Long point requests; like in a chat, we don't want to ping back the server to check if there is more data
Web sockets will do that but IE 6-7 won't ...
* HTTP Parser: no buffer, 28 bytes / HTTP stream : very little memory
* exec some system commands, err / output within a callback
Rails waits for the response while querying the DB
Latency: DISK & NETWORK r 41M & 24M cycles : very very far
calculate is 2 cycles ....
Multithreading is not free, context switching takes memory
nginx does that well
Node.js : callbacks
Single process single thread single execution stack model w/ event callbacks. NEVER block on I/O
js suits very well this style: anonymous functions, closures, 1 callback at the time
The Scottish Ruby Conference is over and to quote a few tweets, that was *AWESOME* !
Here are my organic notes from the conf: mostly people keywords and links I want to remember :-)
Tech Conferences are about socializing and networking. Sadly, the French conferences Paris on Rails 2007 & 2008 completely missed some social time and I hope the French speaking ruby community (who's up for it?) will organize a ruby conf in 2010 or 2011 in a big café or in a venue where people can chat.
I finally understood the point of twitter :-) It took me a while! At the conference use the #scotruby tag to gather, chat, recommend stuff, make some live jokes ... socialize!
Later, he explained why mobile phones with internet access will encourage people to drop their "heavy metal box with 4 wheels to transport 1 person which make a lot of noise, eat lots of public space and pavements and pollute the air with nasty particulate matters" in favor of public transports. OK maybe he used the word "car" instead of metal boxes ....
His point was: why spending 1 hour in a car doing nothing when you can work / read yr emails in the train or bus? When peak oil arises, people who live in area with no public transports will have to move anyway so you'd better anticipate on that one :-)
Therefore, I have decided to go my green "coming out", I will declare publicly for who I want to work, focussing on social and ecological impacts.
I have started writing this blog post in the train from Edinburgh en route to London. There is indeed an excellent free WIFI connection and electricity plugs for every seat! KUDOs to EastCoast for that. For once, I wish the French National Railways follow the example of the British :-) I have finished it in a café inside King Cross International train station, again with free internet!!! Apart from Mulhouse train station, I don't know any other train station with free WIFI :-(
I am definitely impressed by the public service internet provided by the British.
Jim Weirich's keynote: an interesting design idea about using 3 different levels of abstractions: a first level with private primitives and then public methods which use these primitives
cucover, still active Matt?
Ragel will improve parsing time a lot : gem install gherkin
Jo is working on a new Testjour which will work on EC2
test js with harmony
cucumber --format usage, The slowest step definitions (with duration) are listed first
Mocks: just an introduction but with real people playing the mocks: quite fun! Shame I did not go to the BoF session about Mocks to describe the use cases I find totally inappropriate for mocking design development.
RSpec: require 'spec_helper'
To make sure you don't blow up yr app server running rake tasks or scripts; use:
http://github.com/jnicklas/capybara by Jonas Nicklas. webrat is dead, viva Capybara!
http://redcareditor.com/ by Daniel Lucraft, an IDE based on Eclipse / jruby with very little core and many plugins. Driven by Cucumber and RSpec: wow!
http://twitter.com/timbray Key Note was the most inspiring talk for me at this conference. I had to chance to chat with him at the party on Saturday night and I really enjoy it. He divided his talk into 3 parts:
He also mentioned http://1978th.net/tokyocabinet/
Scribd.com is one of the largest Rails sites on the Internet, and at that scale, even innocuous new features or fixes can bring the website down. I'll walk through some of my mistakes and tell you what I've learned about writing code for giant Rails sites.
USe Postgres with a functional index
Use WHERE login = BINARY 'mylogin' for case insensitive
How to leverage speed-coding and technical debt for business success.
where is the Train which will hit you? Don't skip Planning
Start with a Question; How done is done done?
Plugin should work out of the box (copy / paste),
satisfing. timebox 30 min
cheat : sacrifice performance
1995: Ruby & js were born
functions are objects,
Functions as args aka callbacks
this with apply ...
protoype got it wrong
link_to_remote : NOOOOOOOOOOOO bad SoC
test rake tasks
Rails will see defaults set in DB
Genetic Algorithms with Ruby
Readers who reach the end of a book are entitled to some praise
from the author, since they have paid him the greatest
compliment an author can receive. Yet those readers who do reach the
end are probably in less need of rewards than the dropout, for it
would seem impossible to go all the way through any book without
receiving the reward of learning. Perhaps it is possible to go through this book
without being touched, although as one reviewer commented, \"One
comes away with the feeling of having spent a pleasant but somewhat
'wasted' afternoon of reading, and as the old joke goes, 'it ain't till you
try to turn your head that you realize how sharp the razor was.' \"
Although I would have been happier with a less violent metaphor, the
remark captures what the book has tried to do. My students have had the
same reaction. A typical remark made a year after attending a seminar
is: \"It was pleasant enough talking about those things, but then I began
to see what was going on at the office. Wow!\"
The most important thing that this student learned was not any
particular behavioral science result or the name of so-and-so's law to parrot
back on a test. What he learned was that he had been carrying around
with him all his life a well-equipped machine for observing behavior—
but that it had never been used very much. As an anonymous sign at the
computing center put it:
The human mind ordinarily operates at only ten percent of its capacity—the
rest is overhead for the operating system.
So rather than be concerned so much about that computer operating
system, the reader who has really been touched by this book will start to
work on the operating system he carries around in his own central
processing unit—his head. That will be his reward.
But if we make our own operating systems more efficient, if we observe
ourselves and our surroundings more carefully, and if this efficiency and
observation lead us to be more productive programmers, of what use is
it? The stories of the three wishes send us a message from many
cultures, and from the ancient past: \"If you get your fondest wish, what
then?\" If, by psychological, sociological, and anthropological
investigation or by simple heightened awareness we become better programmers,
to what work shall we turn our talents? For if something is not worth
doing, it is certainly not worth doing right.
Is what we are doing with computers worth doing? is what you are
doing with computers worth doing? Because computers are such
fascinating beasts, because programming is such a game, such a joy, we who
program computers are in danger of becoming the unwitting pawns of
those who would use our toys for not-so-playful ends. Can there be any
doubt that if Hitler had computers at his command, one of the first
application would have been keeping closer track on Jews and Gypsies
so that all who should have gone to the ovens did go to the ovens? Can
there by any doubt that if Pilate had computers, they would have been
used to keep the information from informers, the better to crucify those
that were crying out for crucifixion by their heretical zeal? Can there be
any doubt that somewhere in our country today some human beings are
using computers as just another, finer weapon in their arsenal of ways
to subjugate other human beings to their wishes—to their conception of
the proper life of man?
And having said all that, can there be any doubt that such people—
now as in 1939 or at the dawn of the Christian era—find many willing
hands and brains to carry out their work in return for fun and profit? Or
that some of those willing hands will have held this book, to the profit
of their employers?
Many years ago, just a few years after I wrote my first book with Herb
Leeds, I read an article describing experiments in which monkeys were
subjected to various doses of poison gases, evidentally to see how long
it took them to die. The work was done in a laboratory for chemical
warfare research, with the intent, no doubt, of extrapolating it to human
beings. Thirty-six innocent monkeys, as I recall, met their deaths in this
gruesome way so that someday, perhaps, thirty-six million people could
meet their deaths even more efficiently. The article remained in my mind
for months afterward, and indeed it has remained there to this day. By
pure coincidence, I suppose, I was accosted at a meeting by a nice
young man who had gone out of his way to tell me how much he had
learned from our book, how much it had helped him to become a better
programmer. I asked him what sort of work he did, and he replied that
he worked at a laboratory for chemical warfare research.
Afterward, I tried to rationalize my way out of my depression by
imagining that it was a different laboratory, which it might have been, or
that he never worked on the monkey experiment, which might also have
been true. But I knew that somewhere, someone who had learned from
me was participating in such experiments and worse. I knew that I shared
the responsibility—that writing a book is not merely teaching means to
unknown and unimagined ends. For a long time I could not write, perhaps
for that reason, or perhaps for others. But, eventually, my ego got the
better of me and I began again, determined to try making my books
unusable to any but the pure at heart.
To a certain extent, this book may have achieved that goal, for the
idea of the programmer as a human being is not going to appeal to
certain types of people, and they will neither finish the book nor profit from
it. But it is naive, I now realize, to expect that bad systems cannot be
built by people with good hearts. Otherwise, why would I encounter so
many bad systems when almost all of the people I meet are wonderful?
No, something else is needed, something not within the power of an
author to give to a reader. As Malraux once said, \"It is the work of a
lifetime to make a man.\" A book can be but a tiny part of that work—
the rest is up to you, and the work will never be finished.
Having said all that, I do not shrink from personal responsibility for
what I have done in writing this book. We stand at the brink of a new age,
an age made possible by the revolution that is embodied in the computer.
Standing on the brink, we could totter either way—to a golden age of
liberty or a dark age of tyranny, either of which would surpass anything
the world has ever known. Perhaps no individual's efforts will make any
difference in the result, but we must never cease trying, for then the
result is sure to be tyranny. This book is my effort against the tyranny,
the enslavement of men by other men and by their own ignorance. Would
that it not be adopted by the forces of tyranny themselves, as no doubt
it will be. Lacking that hope, I can only hope that its use to the other
forces will, in the balance, be greater.