|
Opinion
Agile Development and the Developer/DBA Connection
by Scott Ambler
It's time for DBAs to update their skillset with "agile" techniques already being adopted in the developer world
Unlike modern developers, who work in an agile and evolutionary (iterative and incremental) manner, most DBAs tend to work in a "serial" manner. But with the potential of service-oriented applications (SOA) to change the nature of software development, DBAs will have to change along with it.
The adoption of "agile" software development techniques can be a significant factor in that process. Not only are developers already working with new technologies such as services, but they're also adopting new development techniques to increase their efficiency. These techniques--Extreme Programming, Feature Driven Development, Test-Driven Development, and Agile Modeling--vary in specifics, but they share an emphasis on the following:
- A cooperative working style between developer and DBA
- Data management techniques that support agile software development (vs. traditional, serial techniques, which do not)
- New, flexible development tools that reflect the realities of modern environments.
In this column, I'll describe some techniques that encourage close cooperation between developers and DBAs during SOA development projects. A few simple changes in the way that you work can turn you into the "go-to" person instead of another "avoid or ignore" one.
Agile Software Development Explained
Agile Software Development (ASD) is a people-oriented, co-operative approach that focuses on high-value activities. Agilists typically work in an "evolutionary" (iterative and incremental) manner, producing working software on a regular basis.
The agile manifesto (Agile Alliance 2001a) defines four simple value statements that are the foundation for agile methods:
- Individuals and interactions over processes and tools. Teams of people build software systems, and to do that they need to work together effectively--including but not limited to programmers, testers, project managers, modelers, and your customers. Tools and processes are important, it's just that they're not as important as working together effectively.
- Working software over comprehensive documentation. When you ask a user whether they would want a 50-page document describing what you intend to build or the software itself, what do you think they'll pick? Doesn't it make more sense to work so that you produce software quickly and often, giving your users what they prefer? Furthermore, I suspect that users will have a significantly easier time understanding any software that you produce over complex technical diagrams describing its internal workings or describing an abstraction of its usage. Documentation has its place; written properly it is a valuable guide for people's understanding of how and why a system is built and how to work with the system. However, never forget that the primary goal of software development is to create software, not documents. Otherwise it would be called documentation development, wouldn't it?
- Customer collaboration over contract negotiation. Working with your customers is hard, but that's the reality of the job. Having a contract with them is important, but it isn't a substitute for communication. Successful developers work closely with their customers, they invest the effort to discover what their customers need, and they educate their customers along the way.
- Responding to change over following a plan. People change their priorities for a variety of reasons; as work progresses, their understanding of the problem and the solution will change. The business environment and the underlying technology change, too. There is nothing wrong with having a project plan--in fact, I would worry about any project that didn't have one. However, a project plan must be malleable; that is, there must be flexibility to change it as your situation changes or your plan quickly becomes irrelevant. If change is a reality, doesn't it make sense to follow a process that reflects this fact, instead of a process that unnaturally attempts to control change in an often ineffective and bureaucratic manner?
Agilists are often criticized for a lack dedication to formalized software processes. On the contrary: agilists do in fact use development tools and follow software processes, it's just that we insist on using tools that add value and processes focused on successful delivery of software--as opposed to overly bureaucratic processes focused on successful delivery of documentation. Agilists do model and write documentation, but we focus on high-value models (many of which are discarded after use) and concise and effective documents. We manage change incredibly well; we just don't see the need to follow an onerous and bureaucratic process to do so.
One of the important points about agile software development is that it doesn't specify any particular type of data-oriented development. There's nothing specific about Java, or COBOL, or web services there either--the values and principles are independent of technology. However, my experience is that the values of the data management community can and should be reflected in the agile toolkit.
This is where the Agile Data (AD) method comes in. It defines six philosophies for agile data-oriented development:
- Data. Data is one of several important aspects of software-based systems; therefore, DBAs should be involved in every development project. However, don't forget that the concerns of object technology, web services, components, and other technologies are also important.
- Enterprise issues. Development teams must consider and act appropriately regarding enterprise issues such as consistent architecture, standards, and tools.
- Enterprise groups. Enterprise groups exist to nurture enterprise assets and to support other groups, such as development teams, within your organization. These enterprise groups should act in an agile manner that reflects the expectations of their customers and the ways in which their customers work.
- Uniqueness. Each development project is unique, requiring a flexible approach tailored to its needs. One software process does not fit all and therefore the relative importance of data varies based on the nature of the problem being addressed. Sometimes DBAs will play key roles on a project, and at other times only minor support roles.
- Teamwork. DBAs and developers must work together to ensure project success; business stakeholders shouldn't have to tolerate inefficiencies resulting from internal IT squabbling.
- Sweet spot. You should actively strive to find the sweet spot for any issue, avoiding the black and white extremes to find the gray that works best for your overall situation. For example, a database using only natural keys is one extreme, using only surrogate keys is another, yet the vast majority of databases contain both--why do people still argue about this sort of issue?
The AD method forms the philosophical foundation upon which developers and DBAs can work together effectively. However, philosophy isn't enough: you also need workable techniques. Let's explore techniques that enable you to take an evolutionary approach to database development.
Defining Evolutionary Database Development
Is this possible? Absolutely, but you have to choose to work this way. You also need to adopt techniques that reflect the realities of modern software development: Agile Model-driven development, object/relational mapping, code refactoring, test-driven development, and evolutionary performance tuning.
Agile Model-Driven Development (AMDD)
AMDD is an approach to modeling in which you create high-value agile models that are just barely sufficient to address the relevant issue. You don't need to create a complete model right up front; instead, you just need to do enough modeling to get you going and let it evolve over time. Effective developers create models to think through the "big issues" and then use other techniques, such as TDD and refactoring, to address the "small issues" of detailed design.
For example, it is quite common for developers to model the high-level services of an SOA but not to document all the details that they require to build the services. The details are then fleshed out through the development of the test cases. The beauty of this approach is that models and tests are each used for what they're best at: minimizing the overall design and documentation effort while ensuring that 100% of the system is tested.
A major feature of Oracle JDeveloper 10g, in my opinion, is its support for modeling. It supports object-oriented modeling with the Unified Modeling Language (UML) and goes beyond it to include user interface (UI), database, and XML modeling. One of AMDD's principles is Multiple Models, stating that you want to know a range of modeling types so that you can use the one best suited for the job. Although UML is an important part of your modeling toolkit it, unfortunately, isn't complete: in particular, it doesn't include standard ways to model either your UI or data. Oracle has done the right thing by extending modeling features that address what developers need here.
Oracle JDeveloper 10g supports UI development with user interface flow modeling via the Struts Page Flow editor. This is a critical capability for business application developers because it simplifies the creation of J2EE-based applications. Although Struts is a great framework, it can be difficult to code to, so I'd much rather use something like the Struts Page Flow editor to simply generate the code. (Then again, I'm lazy.)
Oracle also got its feet wet with UML data modeling in Oracle JDeveloper 10g, something I've been a strong proponent of for years, with the addition of the Database Modeler facility for modeling tables. Why use the UML to create data models instead of simply using the tried-and-true Barker notation? First, the UML is a standard that developers are likely to understand, making it that much easier for you to work together because you share a common modeling language. Second, this could be a good way to get started with UML, opening up new career opportunities.
A critical feature is that you can easily move back and forth between your model and source code, supporting evolutionary development. It also supports AMDD's "Prove It With Code" practice: on the screen every model looks like it will work, but until you've proven it with working code, you can never be sure.
O/R Mapping
Object/relational (O/R) mapping is another critical skill that all developers and DBAs should understand. Modern applications often use object technology such as Java or J2EE on the front end and relational database (RDB) technology such as the Oracle Database on the back end. The implication is that you need to understand how to overcome the impedance mismatch between the two. Luckily, these skills are fairly straightforward but they do take time and experience to learn. (I've worked with Oracle Application Server TopLink for years because it has been, and still is, one of the most effective O/R persistence layers available. TopLink isn't magic--you still need to know what you're doing--but it's as close to magic as you're ever going to get.)
It's important to recognize that because object and data schemas are developed in an evolutionary manner you will clearly need to evolve your mappings over time. Similarly, difficulties in mapping may motivate changes to either your object or data schemas, perhaps even both at once, motivating you to update your models. This is yet another reason to work with a sophisticated and integrated toolset such as the Oracle Development Suite: your tools need to support what you need to do seamlessly.
Refactoring
Code refactoring is a small improvement to your source code that improves its design without adding new functionality. Example code refactorings include renaming a class, moving an operation from one class to another, or removing a parameter from an operation. Simple changes, yet they can require a significant amount of work without tool support. Refactoring enables you to keep your design quality high, reducing your need to create detailed design models--if your programmers refactor ruthlessly then you know a high-quality design will quickly emerge.
JDeveloper has some basic refactoring functions, but for complete refactoring functionality, try the RefactorIT extension from Agris Software AS. RefactorIT is a leading refactoring browser, a tool that allows you to easily refactor your code. For example, when you rename an operation, not only do you have to do it in the implementing source code, but you also need to change the name in every single place the operation is invoked. Refactoring browsers do that for you automatically.
A database refactoring is a small improvement to your database schema that doesn't change its functional or informational semantics. Database refactoring, like code refactoring, enables you to evolve your design over time. Unfortunately, database refactoring is so new that you still need to modify your database schema using your existing DBA skills and tools--the hard part of being an DBA will still be with us for some time. (Note that database refactoring is made significantly easier when access to a database is encapsulated via a persistence framework.)
Test-Driven Development (TDD)
TDD is an approach where you write a new test, you watch it fail, then you write the little bit of functional code required to ensure that the test passes. You then proceed iteratively. The advantages of this approach is that you know at all times that your application works, and it enables refactoring because you can safely make changes to your application knowing that you can identify any breakages resulting from that refactoring. (TDD is a very common approach for agile application developers. For example, the JUnit framework (www.junit.org) can be used with Oracle JDeveloper 10g, and database testing frameworks such as Dbunit (www.dbunit.org) are becoming more and more common.)
Evolutionary Performance Tuning
Because modern systems use several technologies, including object technology as well as RDBMSs, DBAs must be prepared to tune both these technologies and the interactions between them. Because you're delivering working software incrementally--perhaps monthly, weekly, or even daily--you must performance tune on an ongoing basis. The implication is that performance tuning may motivate changes to your object schema, your data schema, or to your mappings between the two. Furthermore, changes to any of these things may have a performance impact, which in turn could motivate an iterative change to another aspect of your system. It's all interconnected.
A Radical Choice
Most developers and DBAs, when asked what they think about their counterparts, say "I love them" or "I hate them"--there rarely seems to be a middle ground. Software development has clearly shifted toward incremental development techniques over the last decade and appears poised to shift even further toward agile techniques in the coming decade. If you want to be one of the technologists that people enjoy working with, then you'll need to adopt agile techniques as well as new tools that enable you to apply them.
Much of this advice may sound radical to you. Due to the cultural divide between the object and data communities, many data professionals missed out on the object revolution from which agile software development derives. Worse yet, many of the "big thinkers" in the data community are woefully behind when it comes to modern software processes, and frankly, they have misled the data community for quite a while.
Whether or not you agree with this observation, it's clear that DBAs need to start adopting the evolutionary techniques that I have summarized here. Otherwise, developers are likely to consider you part of the problem, not part of the solution, and will find ways to ignore you. That doesn't sound like a good career move to me.
Scott W. Ambler is a Senior Consultant with Ronin International Inc. (www.ronin-intl.com), a services firm specializing in software process improvement and architectural consulting. Scott is the author of several books, including Agile Database Techniques: Effective Strategies for the Agile Software Developer (Wiley, 2003) and the forthcoming The Object Primer 3rd Edition: Agile Model Driven Development (AMDD) with UML 2 (Cambridge University Press).
|
Bookmark
