I'd like someone to give me very basic reasons for why I need Behavior Driven Development (BDD, see RSpec) and/or Test Driven Development (TDD, See Test::Unit). I'm a fairly competent programmer. I think I use what they call an "iterative" style of programming; I don't rush out to build the entire program at once. I do a very small framework, then attempt to run it and see if it's giving me the result I expect. If it's not, then I figure out what's going wrong, and attempt to solve it before going on to the next small part. You could say I do several iterations in an hour, to give an idea of time. It looks like TDD does exactly what I'm doing, but introduces far more code to the equation. One of the biggest features of TDD is being able to run a suite of tests to make sure new features don't break old features. Yet I have an eye for this while developing; I either fix the upcoming problem because I know it's going to happen, or I'm doing it wrong in the first place. Sure, I occasionally miss, but who doesn't? I find the issue as quickly as I can and resolve it in a clean style. BDD looks like a way to make it easy for non-programmers to do programming, but not having people keep several functions in mind at any one time. And it also does TDD? Am I close? But everyone continually praises TDD and BDD, which leads me to believe I'm missing something profound that could make me a better programmer. So what's the big deal? Clinton Judy Web Developer Glenn O. Hawbaker, Inc.
on 15.08.2008 22:16
on 15.08.2008 22:35
On Fri, Aug 15, 2008 at 4:10 PM, Clinton D. Judy <cdj@goh-inc.com> wrote: > BDD looks like a way to make it easy for non-programmers to do > programming, but not having people keep several functions in mind at any > one time. And it also does TDD? Am I close? BDD is simply an altered vocabulary for TDD. Vocabulary is important; it effects how you think about a problem and how you decompose it. For a lot of people, including myself, thinking about things in terms of specifications of behavior rather than in terms of "how do I test this method" leads to better-designed code. What's the deal? Like pair programming, BDD/TDD is one of those things you have to try to get. If you can, find someone who is a pro at it and do some TDD coding together. At some point it will "click". For me it was when I spent several weeks coding C++ by myself with nothing but the spec for a communications protocol, and the very first time we integrated my library into the main product it worked out of the box. That kind of integration experience was unheard-of where I used to work. I was hooked from then on, and I've never gone back. Most of the TDDers I know have their own similar conversion story. I find that when I'm writing code test/spec-first, I don't have to worry about debugging. The code is usually right the first time, and when it isn't the problem is obvious. It also forces me to write code that is properly decoupled and obeys the single responsibility principle, making changes much easier. It sounds like you are doing most of your coding alone, on projects that you can pretty much hold in your head at the one time, and on which you can manually test without much trouble. When you start to work on projects that haved tens or hundreds of thousands of lines of code, have had many developers, some of whom are long gone, and which could take days to manually test, the benefits of complete test/spec coverage become impossible to ignore. And the only way to guarantee complete coverage is to write the tests first. But really, the only way to discover the joys of BDD/TDD is to do it for a while. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com
on 15.08.2008 22:36
On Fri, Aug 15, 2008 at 4:10 PM, Clinton D. Judy <cdj@goh-inc.com> wrote: > give an idea of time. I pulled this link off the TDD page on our internal wiki here at work: http://www.railsenvy.com/2007/10/4/how-i-learned-to-love-testing-presentation
on 15.08.2008 22:38
I see your point. I'm struggling with TDD concepts myself. But I
think you answered your own question when you said:
> Sure, I occasionally miss, but who doesn't?
Test programs don't. That thing they tested last week that you forgot
about? They remembered.
I agree that the overhead is extra development time. And I'm sure
test scripts are not infallible -- for example, if you change
functionality in the program but not in the test.
Also, I've said this before here, but as a commercial programmer I've
spent years telling people that the code can't tell you what the
program is *supposed* to do; only what it *does*. Test scripts tell
you what the code is supposed to do. That's pretty cool.
on 15.08.2008 22:52
On Fri, Aug 15, 2008 at 4:35 PM, Shadowfirebird <shadowfirebird@gmail.com> wrote: > I agree that the overhead is extra development time. After doing TDD/BDD for, oh, five years or so, I'm convinced that the "it takes longer" argument is a red herring. It may be true when you're first learning TDD and it takes you awhile to work out how to properly test something. But I've found that once I internalized testing my coding velocity actually *increased*. And more importantly, it became much steadier. Non-TDD projects are often marked by a very fast initial ramp-up, then velocity starts to decline, and then eventually they hit a wall where every change seems to take 2-5 times as long as it should because of all the unintended consequences. TDD projects, by contrast, retain about the same speed from start to finish. Write test; watch test fail; make test pass; refactor; repeat - it's a rhythm that stays constant no matter how large the codebase. I've also found that for me, personally, TDD reduces my initial startup time. In the past I used to spend a lot of time trying to dream up the "right" object model to fit the project, and I'd have a hard time actually figuring out where to dig in and start coding. And then when I did dig in for awhile, it would suddenly occur to me that I'd failed to account for something basic and I'd realize I needed to re-tool my design. And of course the subsequent refactoring would be prone to breakage because I had no regression tests. When I'm doing TDD, all I have to think about to get started is the simplest possible unit of concrete, testable functionality. I make that work the simplest possible way, and then move on to the next simplest case. As soon as I find myself duplicating code I refactor. Out of this rhythm the "right" design emerges, a piece at a time - as it is needed. And there's no wasted time at the beginning trying to figure out how to architect for the future. Which is all to say why I find any project over, say, a hundred lines of code actually goes *slower* without TDD/BDD. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com
on 15.08.2008 23:15
Clinton D. Judy wrote: > I'd like someone to give me very basic reasons for why I need Behavior > Driven Development (BDD, see RSpec) and/or Test Driven Development (TDD, > See Test::Unit). Shine the "Phlip" symbol on the underside of the nearest cloud deck! > I'm a fairly competent programmer. I think I use what they call an > "iterative" style of programming; I don't rush out to build the entire > program at once. I do a very small framework, then attempt to run it and > see if it's giving me the result I expect. If it's not, then I figure > out what's going wrong, and attempt to solve it before going on to the > next small part. You could say I do several iterations in an hour, to > give an idea of time. When you say "attempt to run it", you should be mixing manual testing with running unit tests. They give a second opinion. However, they are permanent. Imagine if you put a "mini-me" in a bottle, who came out and manually tested, instantly, each time you changed the code. Imagine if it would warn if nearly any behavior changed since the last run. You could go faster - making bigger changes - with less manual testing. > It looks like TDD does exactly what I'm doing, but introduces far more > code to the equation. Yet that code should be super-easy to write. A TDD test should obey the Assemble Activate Assert pattern: def test_case foo = assemble_foo() result = foo.activate() assert{ result == 42 } end Compared to production code, that test code looks super-easy to write. Tip: The easier a test is to write, the cleaner and more decoupled your code is. So, counter-intuitively, the more lazy you get writing tests, the more pressure forces your code to decouple. To do TDD, you write the test case first, then write code to pass the test. You only write new code if you have a failing test case. So your code is decoupled before it is even written. > One of the biggest features of TDD is being able > to run a suite of tests to make sure new features don't break old > features. Yet I have an eye for this while developing; I either fix the > upcoming problem because I know it's going to happen, or I'm doing it > wrong in the first place. Sure, I occasionally miss, but who doesn't? I > find the issue as quickly as I can and resolve it in a clean style. You should run a TDD test suite after the fewest possible edits: 10 at the most, and hopefully just 1. A coding session should go type-type Test, type-type Test, type-type Test, in tiny cycles. If you delay to run the test, the more risk you absorb with your edits. This helps sustain projects as they get huge; the code stays just as easy to work with as a tiny project. Nobody can cross-check everything in a huge program. > BDD looks like a way to make it easy for non-programmers to do > programming, but not having people keep several functions in mind at any > one time. And it also does TDD? Am I close? Yes - BDD is TDD with an added layer: A "literate programming" framework that forces you to think in clear English* statements that your client could understand. *or whatever your client speaks! That anyone can do TDD with BDD (and they do) speaks volumes about Ruby's ability to support easy and flexible DSLs. > But everyone continually praises TDD and BDD, which leads me to believe > I'm missing something profound that could make me a better programmer. > So what's the big deal? The big deals are: Almost no debugging, a super-high velocity, the ability to deploy any integration, the ability to rapidly share code with colleagues, code that strongly resists new bugs, and the ability to allow your client to "steer" your project, feature-by-feature, in real-time. If your tests fail unexpectedly, the fault must lie in your last edit. You can undo or revert it. If your tests gate your Subversion or Gitorious version controller, then you know that you can always revert your code to get rid of a test failure. Working in tiny cycles, and constantly integrating your code, allows you to take bigger steps with more confidence. Debugging is the greatest time-waster in all programming. Grizzled senior programmers (like some of my friends!) can all remember working on projects with bugs so big, you could spend the first week just isolating which module contained the bug. With debugging out of the way, you spend your remaining time understanding requirements, implementing them, integrating them, and deploying them. Your project goes very fast as your client learns to request very small features, each an increment over the existing features. Very few features go to waste. And a project with a complete test rig cannot lose its value over time. New code is often easier to write than old code, even despite projects with thousands of features. This metric is unheard of in software engineering, where new features are typically harder to write than the first ones. If your tests gate your integrations, than you could deploy any integration. It might have unfinished features (and you might make their View buttons invisible!), but the tests will tell you it has no bugs. This metric is also unheard-of in classical programming, where a project typically must endure a "polish phase", as much as 25% of development time, to get it "ready to ship". Next, if I need my colleagues to be able to mess with code I wrote (and vice versa), then I don't need them breaking it because they didn't understand it (what are the odds?:). If they run my tests (and they do!), then there's a little bit of me in them, working with them, keeping my features stable while they add theirs. The TDD cycle has three steps: Write a test that fails for the correct reason, write simple code that passes the test, and refactor your code to make it clean and dry. Notice you don't refactor until the code passes its test. Refactoring means merging the new code with the preexisting code. And you don't refactor until your tests say the new code works. This implies only good code with valid features gets merged together. TDD's influence on design cannot be understated: It produces rock-solid code that strongly resists bugs. Under classic software development, you would ask your client for a long list of features, then ask to be left alone for a while! Under TDD, your new features can be reviewed _as_ they emerge. Your client has better odds of getting the features they want if they practice "just in time requirements". They only require features in small increments from those installed features which they like. TDD allows a business cycle as short as a week to add features, review the code, and request for features. This improves the odds your client gets the features they need - not just some of the features they ask for. Your client no longer has the burden of predicting the future and planning every feature they might need. The Ruby on Rails project has copious unit tests; its success is an example of these forces at work...
on 16.08.2008 04:05
On Friday 15 August 2008 15:10:26 Clinton D. Judy wrote: > I'd like someone to give me very basic reasons for why I need Behavior > Driven Development (BDD, see RSpec) and/or Test Driven Development (TDD, > See Test::Unit). BDD is a philosophy and a vocabulary for talking about TDD. They are not mechanically different -- a good TDD suite should support BDD, and vice versa. But they do introduce a few new ways of thinking about the problem. > I do a very small framework, then attempt to run it and > see if it's giving me the result I expect. If it's not, then I figure > out what's going wrong, and attempt to solve it before going on to the > next small part. That's good. Question: How do you know when it's giving you the result you expect? BDD adds an extra step: You write down the result you expect, in code form, as tests. You then write your framework, pretty much as you've described, until the test passes. There are two direct benefits to this approach: First, you know when you're done. If it passes a test and you don't think it's done, write down _why_ you don't think it's done -- again, as a test. In other words, it prevents you from writing a ton of unnecessary code. The second benefit is that, coding this way, you accumulate some small measure of documentation: By reading your tests (which BDD calls "specs"), it's possible for someone to get an idea of what your code is supposed to do. > It looks like TDD does exactly what I'm doing, but introduces far more > code to the equation. Correct. I won't sugarcoat it -- when I'm doing proper BDD/TDD, I will sometimes have twice as much code in tests as I do in the actual program. > One of the biggest features of TDD is being able > to run a suite of tests to make sure new features don't break old > features. Not just new features, but any code change. It means that you can do a massive internal refactoring, or even replace one library (or program) with another, and have some assurance that things still work exactly the way they're supposed to. A simple example: Web browsers. Having the spec, in written form, wasn't good enough. That's why we have the Acid2 and Acid3 tests -- now we can know which browsers have properly implemented the specs, and to what extent. It also provides something for browser developers to work towards. > Yet I have an eye for this while developing; I either fix the > upcoming problem because I know it's going to happen, or I'm doing it > wrong in the first place. The problem is, you know your own code very, very well right now. Will you know this code that well in five years? Will you be able to change any part of it without fear that you will break something? And, perhaps more importantly, will a total stranger be able to sit down and do the same? > Sure, I occasionally miss, but who doesn't? Your test suite. > BDD looks like a way to make it easy for non-programmers to do > programming, but not having people keep several functions in mind at any > one time. It's not about keeping several functions in your mind. It's about keeping the entire program in your mind at once. But the mention of non-programmers does bring to mind another benefit, and one which I rarely see implemented: Stories. A story is a way to express, in plain text, how a program should behave, from the user's point of view. Ideally, a customer could write stories by themselves. Realistically, you would probably write the stories with the customer -- they do read like plain English, so the customer can understand them, but they need to be somewhat stricter, so that you can more easily use them to write tests later. Given a story, and a bit of translation, you can define your eventual target behavior, exactly the way the customer (or your boss) wants. Converted to integration tests, you have feedback as to when you're done -- when it does exactly what the customer wants. I'm still not fluent enough at writing tests that I can actually start with a clean project and test first. I still usually end up doing an evening or a weekend worth of code before I have enough of a program in place that I can properly reason about what my tests would look like -- what I actually expect from this program. But I'm not sure yet whether this is a deficiency in BDD or in my own line of thought. And I do find that after a week or so of development, the program has gotten to a point where I need tests -- or at least, where I'm doing so much testing manually that it's a pain. Simple example: I'm writing a program which spiders a certain website, scrapes some useful information out of the HTML, and dumps it into a database. I've been building it incrementally, as you've described, but every time I want to make sure something works, I have to start with a fresh database, open up an irb console, and have it download the same page. I then have to browse through the database records, making sure they match what I'm expecting. I'm about to convert this process to a test suite -- some pre-downloaded HTML, so I'm not hitting the site every time, and a test to cover this behavior, so that I don't have to keep browsing through records.
on 16.08.2008 04:30
David Masover wrote: > BDD is a philosophy and a vocabulary for talking about TDD. They are not > mechanically different -- a good TDD suite should support BDD, and vice > versa. But they do introduce a few new ways of thinking about the problem. BDD is "customer facing". It's _supposed_ to be about writing the tests in terms a civilian client could understand and agree on. > It's not about keeping several functions in your mind. > > It's about keeping the entire program in your mind at once. And it's about keeping the entire program out of your mind all at once. It's about making changes while looking at a huge application like a map thru a soda straw. I want to make one little change, in one corner, without worrying that I will derail some other far-flung part of the app. Its tests will defend it. Good stuff snipped!
on 16.08.2008 13:55
Hi -- On Sat, 16 Aug 2008, Phlip wrote: > Clinton D. Judy wrote: > >> BDD looks like a way to make it easy for non-programmers to do >> programming, but not having people keep several functions in mind at any >> one time. And it also does TDD? Am I close? > > Yes - BDD is TDD with an added layer: A "literate programming" framework that > forces you to think in clear English* statements that your client could > understand. I wouldn't call it literate programming in the Knuthian sense, though. Knuth says: "The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style."[1] It's a different undertaking from writing BDD specifications (even though of course it's nice if BDD specifications are reasonably well-written). David [1] http://www.literateprogramming.com/
on 16.08.2008 14:07
Hi -- On Sat, 16 Aug 2008, Clinton D. Judy wrote: > BDD looks like a way to make it easy for non-programmers to do > programming, but not having people keep several functions in mind at any > one time. And it also does TDD? Am I close? I wouldn't say that it makes it easy for non-programmers to program. At its best, it can provide output and summaries that non-programmers can easily read, understand, and amend. But the programming part of BDD is truly programming. This is actually at the root of why a lot of people, including me, find it hard to bond with RSpec. In some respects, the syntax tries to look and read like English, but it's still 100% programming and there are serious impedance mismatches between English and Ruby. But RSpec generates summaries that are purely in English, and that describe the program's requirements in such a way that any stakeholder in the project, programmer or not, can read them. I would not expect a non-programmer to be able to read RSpec code. You should, however, check out RSpec if you're interested in BDD. It represents a fantastic achievement (my problems with it are at the level of my own stylistic preferences), and has a huge following. > But everyone continually praises TDD and BDD, which leads me to believe > I'm missing something profound that could make me a better programmer. > So what's the big deal? It scales. If you're doing something small, you can run it and eyeball the results and that's fine (and that is, in fact, a test). Like everything else in programming, though -- calculations, data munging, administrative tasks -- testing your code benefits from being automated when you have to do more than a miniscule amount of it. It's really that simple, and everything else proceeds from that. David
on 16.08.2008 15:58
On 16 Aug 2008, at 12:51, David A. Black wrote: >> that your client could understand. > > I wouldn't call it literate programming in the Knuthian sense, though. > Knuth says: "The practitioner of literate programming can be regarded > as an essayist, whose main concern is with exposition and excellence > of style."[1] It's a different undertaking from writing BDD > specifications (even though of course it's nice if BDD specifications > are reasonably well-written). Many programmers believe they're bad at writing essays, and rather seeing an opportunity to improve their skills they would rather write tests. But tests are only as good as the analysis which goes into them and can very quickly become redundant when working on greenfield projects with very loose requirements. They're also as prone to error as any other code so beyond a certain level of complexity it's not entirely obvious that they improve code quality rather than hindering it. That said, everyone seems to love them so they must have some value. Ellie Eleanor McHugh Games With Brains http://slides.games-with-brains.net ---- raise ArgumentError unless @reality.responds_to? :reason
on 16.08.2008 16:10
Hi -- On Sat, 16 Aug 2008, Eleanor McHugh wrote: >>> could understand. > are only as good as the analysis which goes into them and can very quickly > become redundant when working on greenfield projects with very loose > requirements. They're also as prone to error as any other code so beyond a > certain level of complexity it's not entirely obvious that they improve code > quality rather than hindering it. The essay-writing point, though, isn't at odds with writing tests; it's more about documentation. I agree entirely with your point about complexity -- and also simplicity, as a test like this: assert_equal(my_method(x), 10) can pass based on: def my_method(x) 10 end even if my_method is really supposed to be doing calculations. It's not hard not to do that kind of cheat, of course, but it's a good reminder that there's nothing magical about tests. David
on 16.08.2008 16:37
David A. Black wrote: >> Yes - BDD is TDD with an added layer: A "literate programming" framework that >> forces you to think in clear English* statements that your client could >> understand. > > I wouldn't call it literate programming in the Knuthian sense A> hence the "scare quotes", and B> hence Knuthian literate programming is less than useful for us mortals. If Knuth only intended to invent markup languages in his spare time, and did not intend to split the difference between developer-facing and customer-facing verbiage, then we are beyond literate programming. Here's an example of a test case written in programmer-speak: test_strobe_count_down_fields_in_pre_fight_states Now reverbiage it: specify 'the audience sees an animated countdown before the fight' do Figuring out how to say an internal detail in terms a client can understand is often brain-wrenching. As developers, our brains are prepped with developer terms. The B in BDD stands for "Behavior the client has requested"...
on 16.08.2008 16:54
Hi -- On Sat, 16 Aug 2008, Phlip wrote: > B> hence Knuthian literate programming is less than useful for us mortals. If you want to do what Knuth created it for, it is, or can be, very useful. As far as I know it has nothing to do with testing. > specify 'the audience sees an animated countdown before the fight' do > > Figuring out how to say an internal detail in terms a client can understand > is often brain-wrenching. As developers, our brains are prepped with > developer terms. The B in BDD stands for "Behavior the client has > requested"... My point is simply that "literate programming" is already a known term that refers to something other than creating clear phrases to describe tests. I'm not suggesting any kind of fight-to-the-death (or "it 'fights to the death' do :-) between them. David
on 16.08.2008 17:31
On 16 Aug 2008, at 15:33, Phlip wrote: > prepped with developer terms. The B in BDD stands for "Behavior the > client has requested"... Which of course presupposes there is a clear and direct transformation between what the client believes they want, what they actually want, what they should actually get (which is not always the same thing) and how these can be represented in software given whatever other arbitrary requirements apply. I'm too much of a cynic (i.e. have worked on too many complex projects with multiple stake-holders) to be convinced of this fact, but I definitely wouldn't discourage anyone else from giving it a go as different tools work for different people :) Ellie Eleanor McHugh Games With Brains http://slides.games-with-brains.net ---- raise ArgumentError unless @reality.responds_to? :reason
on 16.08.2008 17:50
Eleanor McHugh wrote: > worked on too many complex projects with multiple stake-holders) to be > convinced of this fact, but I definitely wouldn't discourage anyone > else from giving it a go as different tools work for different people :) That's why you don't T- or BDD in a vacuum. Your "goal donor" should review your features, and their tests, in tiny little increments, every couple of days, with a new release and a new requirements list each week. The "goal donor" writes a user story saying "if you fail to log in six times, put the user's IP in a killfile for one hour", then you should have a spec that says the same thing. Doing exactly what the client says would be "malicious compliance", if letting them follow up weren't so easy.
on 16.08.2008 19:41
On Saturday 16 August 2008 06:51:23 David A. Black wrote:
> I wouldn't call it literate programming in the Knuthian sense, though.
Well, TDD and BDD seem very much against the Knuth style. After all...
"Beware of bugs in the following code. I have only proven it correct,
not
tested it."
on 16.08.2008 19:46
Hi -- On Sun, 17 Aug 2008, David Masover wrote: > On Saturday 16 August 2008 06:51:23 David A. Black wrote: > >> I wouldn't call it literate programming in the Knuthian sense, though. > > Well, TDD and BDD seem very much against the Knuth style. After all... > > "Beware of bugs in the following code. I have only proven it correct, not > tested it." It was: "Beware of bugs in the above code; I have only proved it correct, not tried it." And it was, after all, a warning, not a best-practices recommendation :-) David
on 16.08.2008 19:52
On Saturday 16 August 2008 08:54:28 Eleanor McHugh wrote: > But tests are only as good as the analysis which goes into them > and can very quickly become redundant when working on greenfield > projects with very loose requirements. I would say that such a project is very much in need of some requirements, even informal ones. > They're also as prone to error > as any other code so beyond a certain level of complexity it's not > entirely obvious that they improve code quality rather than hindering > it. Well, I don't really see how they could hinder it... And you're right -- they are prone to error. But the point of a test is to make error noisy -- generally, when you have a bug in your test, you'll know about it very quickly. In fact, TDD suggests that you write the failing test first. That implies that your test has to be at least half-right, when written -- it has to fail. It also implies that by the time a test and its corresponding code has been accepted, the test has failed on at least one case where there was a problem with the code, and succeeded on at least one case where the code is believed to be correct. It's not infallible, no, but it is much better than nothing. And as I said in my other post, certainly by the time you get to integration testing, either you're going to be doing it by hand, or you're going to have it automated. Which would you prefer? > That said, everyone seems to love them so they must have some value. Wrong attitude, and, frankly, Godwinbait. (Everyone seemed to love the Nazis! Did they have some value?) Certainly, if everyone loves them, you should take a second look. But if you really can't find any value in them, it might not be you. It might be that there really is no value in them. I think tests are good and useful, but I don't want you or anyone to be so easily led. The truth is what it is, no matter what other people believe.
on 16.08.2008 21:40
David Masover wrote: > I would say that such a project is very much in need of some requirements, > even informal ones. Here's a nice post by one Scott Ambler (who I suspect is down with BDD): http://www.ddj.com/architect/204200877 He ran a survey comparing projects that called themselves "Agile" to those that called themselves "traditional". One big difference between them is Agile uses "just in time requirements", while "traditional" uses "big requirements up front". Scott then very politely tries to rationalize why the Agile projects claim a higher success rate - despite the survey also stated more folks called their own process "traditional". Put another way, the survey was about traditionalists decrying their own failures - but of course Scott is too polite to come right out and state that. The point of TDD is to reduce the risk that your requirements are wrong. Rapidly delivering small increments of new features, every week, complete with literate test cases, helps keep your client "in the loop".
on 18.08.2008 13:51
On Aug 16, 11:27 am, Eleanor McHugh <elea...@games-with-brains.com> wrote: > Which of course presupposes there is a clear and direct transformation > between what the client believes they want, what they actually want, > what they should actually get (which is not always the same thing) and > how these can be represented in software given whatever other > arbitrary requirements apply. The whole "requirements in front" idea is perfectly simple and logical; also, typically impossible or wrong. "If I'd asked people what they wanted, they would have asked for a faster horse." -- attributed to Henry Ford. Ron
on 18.08.2008 16:07
On 18 Aug 2008, at 12:46, paron wrote: > The whole "requirements in front" idea is perfectly simple and > logical; also, typically impossible or wrong. > "If I'd asked people what they wanted, they would have asked for a > faster horse." -- attributed to Henry Ford. Very true. As my experience is primarily green-field development I've spent my career arguing with clients who not only didn't understand their problem domain but were entirely blinded to that fact by their preconceptions. This is not to say that the requirements they bring to the table are completely worthless, but they have to be viewed in terms of need much more than of wants to develop a cost-effective and lasting solution. Taking the horse analogy, a man who wants to buy six horses may be doing so because he needs to plough three fields in a day and be used to thinking in terms of three teams all running in parallel, when in actual fact a tractor may be able to do those same three fields in a single day by serialising the task and thus do away with the need for two additional teams. That's essentially the point of a faster horse. To convince the man of this you need a working tractor to demonstrate and the only test that matters is the ploughing, which is a good argument for prototyping. All other tests that you may frame for the tractor - or for the horses for that matter - fall under the general category of robustness/quality and will generally be ignored by the man at this stage unless the tractor breaks down during the demonstration. From what I've seen of BDD and TDD in practice, they focus a lot of effort into making the teams of horses as close a match to those the man wants as possible so as to achieve incredibly high satisfaction ratings: they're thus an excellent marketing technique. However what the man fundamentally needs is a more effective ploughing solution that costs him less money, and figuring that out requires a broader perspective than that provided by just-in-time requirements analysis. He probably has to put more effort into getting a tractor - as reflected by a higher capital investment and higher fuel costs - but the productivity of this solution more than pays for the extra investment. Would he be satisfied watching the whole tractor construction project from initial back-of-the-envelope design to real- world deployment? I very much doubt it. What he's interested in is ploughing, not engineering. But what he ends up with is still the better solution to his actual problem than the wonderfully supportive development of a six horse ploughing solution even though that's what he wanted all along. I can see that BDD + inspiration could develop the tractor as a solution to the problem, but could BDD on its own? How flexible is it in practice and can a project that's 100k+ SLOC of actual application (plus whatever overhead in test code) take revolutionary departures? Does it suit the traditional hacker habit of forking early and forking often? And what are the implications for all that test code? Ellie Eleanor McHugh Games With Brains http://slides.games-with-brains.net ---- raise ArgumentError unless @reality.responds_to? :reason
on 18.08.2008 16:08
On Fri, Aug 15, 2008 at 11:12 PM, Phlip <phlip2005@gmail.com> wrote: > You should run a TDD test suite after the fewest possible edits: 10 at the > most, and hopefully just 1. A coding session should go type-type Test, > type-type Test, type-type Test, in tiny cycles. Some believe that no editing at all shall be done other than making a failing test succeed. I quite adhere to this philosophy, I am however taking the liberty to omit the failing test run before editing as my experience has shown me that enough failing test runs will follow anyway ;) However I do not code unless I want a new test case to succeed, that is only when I am doing TDD of course. Cheers Robert
on 18.08.2008 16:16
paron wrote: > The whole "requirements in front" idea is perfectly simple and > logical; also, typically impossible or wrong. > "If I'd asked people what they wanted, they would have asked for a > faster horse." -- attributed to Henry Ford. Tx I am so passing that one to the XP mailing list!
on 18.08.2008 16:27
On Mon, Aug 18, 2008 at 10:01 AM, Eleanor McHugh <eleanor@games-with-brains.com> wrote: > From what I've seen of BDD and TDD in practice, they focus a lot of effort > into making the teams of horses as close a match to those the man wants as > possible so as to achieve incredibly high satisfaction ratings: they're thus > an excellent marketing technique. However what the man fundamentally needs > is a more effective ploughing solution that costs him less money, and > figuring that out requires a broader perspective than that provided by > just-in-time requirements analysis. I think you may be conflating two different scenarios. There's the scenario where a customer knows approximately what they want, and an agile team can help them triangulate in on exactly what they need with a minimum of wasted effort. Then there's the scenario where someone who has worked in a particular domain for awhile realizes that things could be done in a much better way, and starts a project to implement it. Often, this is someone who has worked on several teams such as previously described. In this case, the visionary, not the projected end-user, is the "customer", at least in initial development. If all goes well, and they hire a good marketing team, the visionary and her team then convince their end-users that what they REALLY want is this revolutionary product. This goes a lot easier if the team in question is already well known and respected for their *evolutionary* work in the same field. I would submit that when a customer hires you to build them a better plow, you should build them a better plow. However, once you have done that a few times you are in an excellent position to go off for six months and innovate a new hydroponic gardening system and sell it to your existing customers. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com
on 18.08.2008 18:48
On Mon, Aug 18, 2008 at 4:11 PM, Phlip <phlip2005@gmail.com> wrote: > paron wrote: > >> The whole "requirements in front" idea is perfectly simple and >> logical; also, typically impossible or wrong. >> "If I'd asked people what they wanted, they would have asked for a >> faster horse." -- attributed to Henry Ford. > > Tx I am so passing that one to the XP mailing list! Why be offended, our planet might be better off with faster horses, I actually would like horses running at Warp 10! (No that is not 42 times 10 square times 12 square, 10 is fast enough). Holding on to them might prove difficult though. Well did we get off topic here? R.
on 18.08.2008 21:57
Thanks for the great replies. I'm attempting to learn BDD through rSpec now. Actually, are there any suggestions for learning rSpec for someone who's never done BDD or TDD before? I bought the first rSpec video tutorial from Peepcode, but haven't touched it yet.
on 18.08.2008 22:07
On Mon, Aug 18, 2008 at 3:54 PM, Clinton D. Judy <cdj@goh-inc.com> wrote: > Actually, are there any suggestions for learning rSpec for someone who's > never done BDD or TDD before? I bought the first rSpec video tutorial > from Peepcode, but haven't touched it yet. Peepcode's pretty awesome, I'd say go with that :-) -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com
on 19.08.2008 06:10
Clinton D. Judy wrote: > Thanks for the great replies. I'm attempting to learn BDD through rSpec > now. > > Actually, are there any suggestions for learning rSpec for someone who's > never done BDD or TDD before? I bought the first rSpec video tutorial > from Peepcode, but haven't touched it yet. Yes - learn TDD, because that's what most of the tutorials will cover. And Test::Unit comes bundled with Ruby. At the entry level, if you try to learn BDD first, it's just TDD with inverted syntax. Then, also look at non-Ruby TDD tutorials, such as /Test Driven Development/ by either Kent Beck or Dave Astels...
on 19.08.2008 13:05
On Aug 18, 10:23 am, Avdi Grimm <a...@avdi.org> wrote: > > team then convince their end-users that what they REALLY want is this > -- > Avdi > > Home:http://avdi.org > Developer Blog:http://avdi.org/devblog/ > Twitter:http://twitter.com/avdi > Journal:http://avdi.livejournal.com Yes, there's a fundamental difference between making incremental changes and reimagining the problem domain. My background, like Ellie's, is in greenfield situations, which colors my thinking. As she and Avdi point out, BDD is fine for evolutionary changes, but not for revolutions in thinking. BDD is a caterpillar tractor -- controlled and unstoppable. Real innovation is more like dynamite. Even a non-working prototype will help your customer make the leap to see his problem in a different light. It also helps if they are desperate. If they aren't desperate, they will stick with various combinations of horse until they can no longer do so. Then they might be willing to make the leap -- but you still have to show them a prototype. Once you have buy-in to a fundamental change or a novel idea, THEN BDD comes into its own.
on 19.08.2008 16:32
On Tue, Aug 19, 2008 at 7:01 AM, paron <rphillips@engineer.co.summit.oh.us> wrote: > Yes, there's a fundamental difference between making incremental > changes and reimagining the problem domain. My background, like > Ellie's, is in greenfield situations, which colors my thinking. As she > and Avdi point out, BDD is fine for evolutionary changes, but not for > revolutions in thinking. BDD is a caterpillar tractor -- controlled > and unstoppable. Real innovation is more like dynamite. I think you missed my point. BDD is equally applicable to both status-quo development and to radical departures and prototyping. What changes is not the development practices, but the *customer* who helps determine what the proper specs should be. When you are building a typical evolutionary system, the customer is the client; the end user. When you are developing an innovative new product, the customer is *you* - or whoever has the vision for the new product. BDD/TDD is still equally important, but the person defining the specs differs. There is a line of thinking that says that TDD/BDD is inapplicable to prototypes, because the prototype will just get thrown out anyway. In my experience, though, this rarely happens - in the real world you don't have *time* to throw out the prototype, and it winds up getting pressed into service as the basis for the final product. Which then necessitates a lot of painful retrofitting of test coverage. Agile methodologies do generally accept creating "spikes" without testing up front. A spike is basically an exploratory proof-of-concept for one particular high-risk part of the application - e.g. "let's do a spike to prove that we can process 1000 SOAP messages per second". Spikes are not prototypes, however. Their scope is much smaller, usually taking only a few days to a week or two to create. As a result they are a lot easier to throw away than a working prototype. > Once you have buy-in to a fundamental change or a novel idea, THEN BDD > comes into its own. In my experience BDD/TDD comes into it's own as soon as your code exceeds somewhere around 100 lines of code. Or less. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com
on 20.08.2008 06:19
On Monday 18 August 2008 06:46:46 paron wrote: > "If I'd asked people what they wanted, they would have asked for a > faster horse." -- attributed to Henry Ford. That's actually a really good analogy -- but you see, Ford's leap wasn't abandoning what his clients wanted altogether. People obviously do want cars. But why do they want cars? What's the benefit of cars? It would not be very hard to write, up front, a requirement that it be able to transport people and some limited amount of cargo, and that it should go faster than a horse. More importantly, I don't think BDD (or TDD) is about specifying this requirement up-front, nor do they prevent emergent phenomena. It's more about specifying it before you code it. Which could be down to several minutes before you code it. Unless you really are blindly banging around on your keyboard, to see what happens. Which I'll admit to doing -- but I tend to do that in irb, not in source code.
on 20.08.2008 06:33
On Tuesday 19 August 2008 06:01:38 paron wrote: > Yes, there's a fundamental difference between making incremental > changes and reimagining the problem domain. Yes, there is. But it's not a difference in testing methodology. I would use Ruby for any program I could -- I find it equally useful for experimental greenfield apps and aging production apps. Similarly, I would use BDD as soon as I could, in an experimental greenfield app. And I do. That's my opinion, of course. I'll never understand why, but some people prefer to use Ruby or Python to prototype an app, then throw away the prototype and start from scratch in Java or C++. > Even a non-working prototype will help your customer make the leap to > see his problem in a different light. That is true. However, properly-implemented BDD could (sometimes, at least) lead to that prototype faster. All else equal, a working prototype is better than a non-working prototype. I'd also argue that a non-working prototype carries several other, hidden risks -- mainly, the risk of promising too much. No matter how many times you tell them "This is just a prototype; it'll have to be rewritten from scratch; we can't actually do this for another six months..." They're still going to come out of that presentation thinking "Man, that's an awesome prototype! I can't wait!" No, you show the awesome-but-nonworking prototype to your boss. Not even that -- if you can, you show him a whiteboard sketch of it. But even with bosses, and especially with clients, remember the Scotty philosophy: Underpromise, so you can overdeliver. That's how you earn a reputation as a miracle-worker. > It also helps if they are > desperate. If they aren't desperate, they will stick with various > combinations of horse until they can no longer do so. Then they might > be willing to make the leap -- but you still have to show them a > prototype. If you've got all the time it takes them to figure out that various combinations of horse won't work, you have the time to do it right. The first car that Ford showed the world didn't explode. The Wright Brothers built a prototype plane, and it flew. They weren't complete, but for some set of requirements (aka specs/stories), they were working.
on 20.08.2008 14:26
David Masover wrote: >> "If I'd asked people what they wanted, they would have asked for a >> faster horse." -- attributed to Henry Ford. > > That's actually a really good analogy -- but you see, Ford's leap wasn't > abandoning what his clients wanted altogether. People obviously do want cars. The car industry also did it incrementally. The original "horseless carriages" were exactly that - a carriage frame with an engine strapped into the luggage rack. Model T cars look square and funny because that carriage was still there, underneath. After people stopped thinking they needed faster horses, and after they participated in reviewing the designs of cars, then the horseless carriage morphed, feature by feature, into modern cars. > More importantly, I don't think BDD (or TDD) is about specifying this > requirement up-front, nor do they prevent emergent phenomena. TDD allows you to safely change your design and features while keeping out the bugs. That makes just-in-time requirements safer.
on 20.08.2008 19:12
the counter point: TDD seems badly suited to *really* rapid prototyping. I often use TDD for production code that I need to work long after i have moved on, or for production apps that many people need to work on. but its kind of like red-tape. if you dont trust yourself or others, then get a contract. if you do, then word of mouth is fine. also its an issue of scale - if you are working on sth for > 1 week, you need tests to keep it all together. but many proofs are done over a weekend or a hectic all-nighter. if you are looking for a proof of concept - if something works or not (today, and you don't care about tomorrow) then writing iron-clad tests for everything seems redundant. if you have a level of confidence, you can fly blind, and i feel get into a much better state of flow without tests. a week later things may not work as planned tho. additionally tests prove things on a very abstract level. if the wright brothers had spent a bunch of time with RSpec they would have theoretically proved their model of a cylinder would have enough horsepower to lift it itself. and the tests would ensure that the next change to the air system, would be guaranteed to not break the cylinders. good, solid, mainstream engineering. but they would never have captured people's imaginations as much as a working flying machine. if it blew up in a future flight, at least it flew first... /dc
on 20.08.2008 19:54
On Wed, Aug 20, 2008 at 1:08 PM, d c <lister@pikkle.com> wrote: > additionally tests prove things on a very abstract level. if the > wright brothers had spent a bunch of time with RSpec they would have > theoretically proved their model of a cylinder would have enough > horsepower to lift it itself. and the tests would ensure that the next > change to the air system, would be guaranteed to not break the > cylinders. good, solid, mainstream engineering. You mean "Unit Tests", right? The term "tests" encompasses manual tests, unit tests, functional tests, integration tests, usability tests, user acceptance tests, stress tests, fuzz tests, smoke tests, final acceptance testing, etc. I don't know where people get the idea that because you are doing Unit Tests or BDD, you must not be doing any other kind of testing - for instance, firing the app up and using it. TDD/BDD is a great way to drive your coding, but it's not the only kind of testing in a healthy project. See also http://railspikes.com/2008/7/11/testing-is-overrated -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com
on 20.08.2008 20:00
On Wed, Aug 20, 2008 at 1:08 PM, d c <lister@pikkle.com> wrote: > if you are looking for a proof of concept - if something works or not > (today, and you don't care about tomorrow) then writing iron-clad > tests for everything seems redundant. if you have a level of > confidence, you can fly blind, and i feel get into a much better state > of flow without tests. a week later things may not work as planned > tho. I understand the sentiment, but if you feel the need to write iron-clad tests up front, you're probably not going to see most of the benefits of TDD. For me, the flow isn't broken if I limit each test to a single, basic change that I'm about to make. I don't even begin to think about edge cases until I write tests that fail for the bare minimum basics (a single use of a single function). Then I write that code. Then I move on to the next thing. Often times, figuring out what tests I need to write next involves toying around with a prototype, or writing parts of my function in IRB. This helps me figure out how my code should work without having to think it through mentally. Once a get a sense of how that stuff should work, I might refactor a bit under the green light to make room for the new behaviour. Then I write the new failing test, then I write the code. Most of the time, this doesn't cause any noticeable decrease in my velocity, it even increases it sometimes. Purists might say you shouldn't write any code at all before your tests, but I think what's important is that you shouldn't *integrate* any code before you write the tests. Hacking around with an experiment, so long as it's tiny and can easily be followed by a formal test case, is always helpful. -greg
on 20.08.2008 20:57
>> additionally tests prove things on a very abstract level. if the >> wright brothers had spent a bunch of time with RSpec they would have >> theoretically proved their model of a cylinder would have enough > > You mean "Unit Tests", right? The term "tests" encompasses manual > tests, unit tests, functional tests, integration tests, usability well, i guess you could say looking at things in a browser is part of "testing" but thats not the point of this discussion? i came to ruby cos i like to realize ideas quickly. premature test cases is like premature optimization. making a change means fixing a bunch of failing tests... in the first 24 hours of a project, where you *know* things are likely to change a lot, then testing + coding is close to 2X the work. if you're working on a billing system for a bank, or a demo for a flying pumpkin that blows up at midnight, the needs are completely different. but i think the prevailing opine that tests somehow give you more freedom is relative to your event horizon. /dc
on 20.08.2008 21:03
On Wed, Aug 20, 2008 at 2:53 PM, d c <lister@pikkle.com> wrote: > i came to ruby cos i like to realize ideas quickly. > premature test cases is like premature optimization. > making a change means fixing a bunch of failing tests... > in the first 24 hours of a project, where you *know* things are likely > to change a lot, then testing + coding is close to 2X the work. I don't have a problem with doing 24 hours worth of exploratory coding sans tests. But if I decide at the end of that I decide I want to continue, my best bet is to THROW IT AWAY and rewrite it test-first. If you and/or your organization have the discipline to a) stop prototyping early; and b) throw the prototype away, then test-less prototyping is workable. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com
on 21.08.2008 04:07
On Wednesday 20 August 2008 12:08:01 d c wrote: > the counter point: > > TDD seems badly suited to *really* rapid prototyping. I thought so, too. And I got quite a lot done, very fast, with this hobby project I'm working on... Then it reached a threshold -- not a very high one -- where I could no longer reasonably develop the app, because of how long it was taking me to test it, manually. At this point, I at least needed a testing framework. And that was short of having the working prototype that I want. It's obvious at this point that from here on (till the working prototype), not having tests will slow me down, not speed me up. Now, I suppose it depends on the complexity of the prototype. If you want to prototype absurdly fast (a few hours or days), and it's a reasonably simple concept (a couple hundred lines or so), then yes, it's probably faster to just code it. But the longer you put off writing those tests, the worse it's going to be when you finally start.
on 21.08.2008 04:50
David Masover wrote: >> TDD seems badly suited to *really* rapid prototyping. Confer "Architectural Spike". That means a prototype where your colleagues are the demonstration's audience. You should write enough of a new module to "close the loop". The new code will spike thru a view, middleware, back-end, a little productization, and a little testage. Y'all will then rewrite the spike via TDD. However, if the prototype's audience is client-side, they might point here and there, and say, "okay, keep this here, but change that around. In one hour." If they say that, you are already around the bend. The tests must already be in place to sustain the rate-of-change your client now expects of you. This is why, in some circles, the ideal is "there are no Prototypes!" > I thought so, too. And I got quite a lot done, very fast, with this hobby > project I'm working on... > > Then it reached a threshold -- not a very high one -- where I could no longer > reasonably develop the app, because of how long it was taking me to test it, > manually. At this point, I at least needed a testing framework. Right. The bigger the project, the longer it runs, the more you need tests. They help you add new features faster than you added old ones. For example, at work, last year we added a bad database field that got stale very soon. Back then, the code was too fragile to just yank it out. But since then, we have relentlessly tested, and cleaned all around that variable, and today we took it out with no issues. Under Brand X, the sooner you take that variable out, the less you suffer. The code will ossify around it.