ChannelWorld.in, Strategic insights for solution providers

Feeling like your business intelligence effort is a bit sluggish and out of touch with what the business needs? Maybe it’s time to try agile BI, a rapid development methodology that solicits end-user input early and often, and delivers BI systems fast.While the use of the agile software development methodology is a big component of agile BI, it’s by no means its only attribute, says Boris Evelson, an analyst at Forrester Research.

Forrester recommends adopting multiple best practices and next-generation technologies to make BI more flexible. The research firm defines agile BI as an approach that uses processes, methodologies, organizational structure, tools and technologies to help strategic, tactical and operational decision-makers be more flexible and more responsive to the fast pace of change in business and regulatory requirements.

Before developing an agile BI strategy, a company must adapt its organizational structure and enterprise culture for agility, Evelson says, pointing out that no technology or process can address BI challenges if a company’s organizational structure and culture aren’t already designed to be agile. Once an organization is aligned for agility, it’s ready to adopt agile BI processes, he says.

“Very few organizations have implemented agile BI as Forrester defines it,” Evelson says. Based on anecdotal evidence and discussions with many clients, he estimates that out of all the organizations that use BI applications, probably less than 20% of the BI user population within those organizations is leveraging some kind of agile BI. But he predicts that that figure will climb to about 80% in the near future.
Despite the low adoption rate, companies in a variety of industries — including healthcare, retail, education, biotechnology and financial services — are already benefiting from agile BI, according to Evelson.

A Good Fit

“Business intelligence is an enterprise application that is particularly well suited for agility and the agile development methodology,” says David White, an analyst specializing in BI at research firm Aberdeen Group.Research that Aberdeen conducted in 2010 shows that despite the wealth of experience that many end-user organizations have in implementing BI systems, only 43% of business intelligence projects were delivered on time or early. Many companies still struggle to deliver the right information to the right business managers at the right time.

Research conducted by Aberdeen in February and March this year indicates that organizations face three significant challenges when it comes to effectively delivering BI that is truly valuable to the business. One is that data volumes and the number of BI data sources are growing. Another is that the amount of time managers have to make decisions is shrinking. Third, is that demand for management information is either increasing or changing.“One of the most important steps companies must take to ensure that their BI implementations are agile is to analyze their organizations’ business needs and take those into account when BI software is being configured and deployed,” White says.

To determine which organizations have the most agile BI systems, Aberdeen evaluated factors such as the availability of timely management information, average time required to add a column to an existing report, and average time required to create a new dashboard. Another key to success is getting end users heavily involved in the BI development process. At nearly 70% of the organizations that are successfully deploying agile BI, IT and business people collaborate frequently, according to Aberdeen. Such interaction is less common at organizations that are less agile; only 50% of them report frequent collaboration between IT and the business.

The close collaboration that’s needed might not come easily, White notes. “There has always has been a communications barrier between IT and the business,” he says.

White cites the development of a new dashboard to illustrate how collaboration might work: An IT developer planning a prototype dashboard could ask business managers what they need in the dashboard, develop a version, then go back to managers to get feedback — in some cases multiple times.

Rapid Delivery

Organizations using agile BI are tapping into that power. One example is Kiva, a San Francisco-based nonprofit that arranges person-to-person loans via the Internet to help alleviate poverty. In 2010, Kiva implemented a new data warehousing and BI application based on Pentaho’s Agile BI technology to handle its rapidly growing stockpile of data and replace an increasingly inefficient ad hoc analysis process that was done via spreadsheets.

“The nonprofit has used Pentaho’s open-source development platform to create monthly, weekly and program-specific dashboards that product managers and finance specialists can use to track and manage key metrics such as transaction volumes, new user registrations and promotion performance,” says Greg Allen, business analyst at Kiva.

The biggest benefit that Kiva’s agile BI efforts have yielded is the ability to rapidly and incrementally deliver information to end users. Allen says. “Instead of forcing the organization to wait for a ‘big switch,’ we are able to focus on specific business areas to design, create and release data marts and reports as part of our iterative development.”

Allen says having greater agility with BI development gives the organization more flexibility. “Agile BI enables us to capture new data and transform it into information,” he says. Kiva handles millions of financial transactions, and accuracy and consistency are key. “Being agile means finding ways to break large, complex solutions to reporting challenges into iteration-sized chunks so we can measure progress along the way,” Allen says. Having end users involved in the process is vital.

“While being agile allows us to respond to those needs quickly and also sets up the potential for significant reloading time if refactoring is needed. ”But there can be challenges with agile BI development. “Large BI projects can get stalled living in internal cycles focused on minute design details,” Allen says.

Although Kiva is at the beginning of an ambitious plan for its data warehouse and reporting capabilities, “we are already able to deliver new insights on our user base and partners,” Allen says. “Often, early or narrow views into business areas have generated new questions that improve the next iteration of development.”