This data is typically sparse and hard to analyze. The formatting may not be consistent or even well-structured. And, worst of all, it is very hard to tell which events are relevant and which ones are not.

Even if you found a way to consolidate all this information, it will not have any financial meaning and the data is not going to be easy to translate into an adequate business impact.

You may find that you have 2,391,369 subscribers doing an average of 126 “interesting” events per day. So what? This doesn’t tell you much.

You may try to look at your ERP application for some insight, and there is good information there. But ERP applications have very coarse transaction data and you will find out that you cannot go very deep in your analysis. Maybe you’ll see that you have 20 revenue entries this month for the different products and services that you offer. You may discover that products A and B account for 80% of your revenue… so they are very important. You want more details about this! Where do you go?

The answer should be your billing application. By definition, billing collects all kinds of relevant, customer-related events and translates them into charges and invoices. The beautiful thing is that billing not only knows the details about the events… it has the details about your customers! After all, you need to know things like where they are geographically located (I hope you have this in your billing system… or you may have a slight collections problem in your hands). You also should have information about their organizational structure and subscription history.

So billing is the place to look for useful insights about your revenue and opportunities to maximize it. A good billing solution should show you, for example:

• That Customer ABC has less subscribers than XYZ… yet ABC’s subscribers have an average of 500 events per day while XYZ subscribers only do 100. Now you can call ABC to find out all the ways in which they find your service useful. Then call XYZ and pass along those ideas.
• How are your products used in different geographies
• Product trends: seasonality, adoption, correlation with marketing campaigns.
• Subscription trends: which packages are customers signing-up to, which ones are being terminated, upgrades, etc. etc.

Billing is normally seen as just a financial function that belongs to A/R. But if you want to make the most out of your existing Big Data repositories, you’ll want to make sure that your billing provider has the ability to consume a high-volume of transactions and that it has the business analytics features to give you the insights required to improve your business.

 

The explosion of social media, the cloud, and smarter mobile devices has introduced a need for handling more data than ever before. Big Data is a very popular subject, and a lot of interesting technologies are being created to deal with it. One of the most interesting areas of development has been around data persistence. The most important trend: the rise of the “dumb database”. A successful Big Data application has to be designed in such a way that its storage infrastructure is focused on one thing, and only one thing: Persist and Retrieve data.

Popular enterprise RDBMS (Relational Database Management Systems) like Oracle and SQL Server come packed with a number of features that do a lot more than simply persist and retrieve data. They provide the ability to implement logic in different forms (stored procedures, functions, triggers, and elaborate constraints). The power of these tools is almost enough to build an entire application in the database itself.

Another powerful feature includes the query optimizer, and all the different artifacts that support it: indexes of many types, materialized views, statistics, etc. Query optimizers will find the best approach to retrieve the data for a specific SQL query, without the query itself indicating how to retrieve the data… only what data is needed. Query optimizers aren’t perfect, and for certain scenarios RDBMS also offer a capability known as “query hints”. These allow a developer to actually tell the RDBMS how to fetch the data. The use of query hints normally requires additional testing and maintenance, since overriding the default query optimizer will mean that the system will not adapt its retrieval strategy as data grows. It also requires very specialized developers and DBAs, with many years of experience in the RDBMS.

The approach for dealing with Big Data is very different: All the features mentioned above go out the door. Major No-SQL systems that are designed to deal with Big Data simply offer the basic features to save some data, and then retrieve it at a later time. There are normally two indexes to retrieve data in an efficient manner. Anything beyond that is not supported out of the box. There is no concept of a stored procedure or a function, and so on. In exchange for the loss of all these features, No-SQL systems allow unlimited scalability by leveraging inexpensive hardware and by replicating data to avoid any loss in case of hardware failure.

This will sound very scary to anyone who has worked with RDBMS for years, and has come to rely on all these features (I know it was scary to me!).

But, there are good news: There are good alternatives to many of these features. If implemented properly, they will further enhance the ability of your application to scale to Big Data dimensions. In addition, the switch to a No-SQL database does not have to be an all-or nothing proposition and can be done over time.

Some suggestions for dealing with the lost features that you normally get from an RDBMS:

• Functions, stored procedures and triggers: These should be implemented in the application server, and should be written in a modern object-oriented language. Trust me, most of the logic that they implement belong there anyways. This move will improve performance and scalability (no more trips to the database just to execute some logic that happened to be implemented there). You will also be able to apply modern agile practices like TDD (Test-Drive-Development) and automated testing. These will improve code quality and reusability.
• Lack of SQL language: Depending on the No-SQL technology that you are using and the language in which you develop your application, you are likely to find yourself writing 3GL code to access your data. This is not a lot worse than having to annotate SQL with hints, or having to write stored procedures with cursors… which you are likely to do for very large data sets anyways.
  • One bright side is that many languages are starting to embed a domain-specific language that will feel like SQL and can help reduce the amount of code necessary to retrieve data from NoSQL systems. For example, Microsoft .NET supports LINQ (Language-Integrated Query) which can be used with RavenDB.

 Some final suggestions to transition from a purely RDBMS application to No-SQL:

• Start by moving any logic from the database to the application. You essentially want to end up with a schema that only contains data objects like tables and indexes.
• Move complex constraint validations to the application layer. The only type of constraints you want to keep in the database are Primary Keys, Unique Keys, and Foreign Keys*.
• If you use sequential identities in your RDBMS tables, consider changing your keys to use GUIDs. The conversion is not straight-forward, but retrieving an identity from a centralized database server is expensive when you are planning to insert millions of transactions across distributed application servers.**
• When you are at the point of only leveraging a “bare-minimum” RDBMS, you will be in a good place to start moving tables to a No-SQL system. Start with high-volume tables.
  • Be aware that No-SQL databases normally don’t have a pre-defined schema where columns and their data types are declared in advance. This can be seen as a new feature if you have records that can look different from one another. But a pre-defined schema can help detect errors as soon as a record is being inserted into the database, and detecting errors earlier is always a good thing. Your development team should be aware of prior schemas as your application evolves, and they should code the application to be backwards-compatible to allow older records to be used.
  • It is ok to stop at a point where the largest tables have been moved, and you still have smaller tables in the RDBMS. You will be able to enjoy the benefits of a highly-scalable database for those tables, and the remaining nice features of an RDBMS (like Foreign-Key constraints, query optimizer) for lower volume objects (this strategy known as Polyglot persistence).

* I would also recommend using constraints to validate “Enumerator” columns: The kind that has a pre-defined list of possible values. So if you have a “Color” column, check that the value is “Yellow”, “Red”, “Blue”, etc.

** Yes, there are smart strategies to distribute the generation of these. But they normally have limitations or introduce operational challenges. GUIDs are becoming a standard as identifiers in the cloud for this reason.

Out at All About the Cloud, Metanga just left The CODiE Awards 2012 presentation by SIIA and Grant Thornton. We want to be the first to announce and congratulate the CODiE 2012 winners publicly. The energy was awesome and the solutions fantastic.  

Congratulations to all the CODiE 2012 winners:

Best Asset Management Solution:

• FlexNet, Flexera Software

Best International Data Protection Solution:

• Druva inSync Cloud, Druva

Best Software Development Solution:

• Coverity Static Analysis 5.5, Coverity Inc.

Best Use of Open Source Technology:

• Jaspersoft Business Intelligence (BI) Suite, Jaspersoft

Best Database Innovation: 

• Tableau 6.1, Tableau Software

Best Web Services Solution:

• AtomSphere, Dell Boomi

Best Electronic Commerce Solution:

• ShopVisible Ecommerce, ShopVisible, LLC 

Best Supply Chain Management Solution:

• IQNavigator.com, IQNavigator, Inc.

Best Document Management Solution:

• SpringCM, SpringCM

Best Project Management Solution:

• Innotas Project Portfolio Management (PPM), Innotas

Best Systems Management Solution:

• BoxTone, BoxTone

Best Human Capital Management Solution:

• Talent Acquisition Platform, Forte HCM Inc.

Best Content Management Solution:

• LP Marketer, LivePerson, Inc.

Best Relation Management Solution:

•  Coveo Insight Solutions for Customer Service, Coveo

Best Financial Management Solution:

• Xactly 7.2, Xactly Corporation

Best Digital Rights Management Solution:

• Sentinel Cloud, SafeNet, Inc. - SRM Group

Best Mobile On-Demand Application:

• Citrix GoToMeeting for iPad, Citrix Online

Best Mobile Operations Management Solution:

• Good For EnterPrise, Good Technology

Best Securty Solution:

• Director, Venafi

Best Integration Solutions:

• Operative.One, Operative
• Pervasive Data Integrator, Pervasive Software - Integration Business Unit

Best Marketing/Public Relations Solution:

• Spark by Marketo, Marketo

Best Collaboration/Social Networking Solution:

• Gust, Gust

Best Cloud Infrastructure:

• FireHost Secure Cloud Hosting, FireHost

Best Cloud Platform:

• Apprenda - V2.5, Apprenda 

BestBusiness Competitive Intelligence Solution:

• Tableau 6.1, Tableau Software

Best Coud Application/Services:

• Google Apps For Government, Google
• Keynote Cloud Application Perspective, Keynote Systems, Inc.
• Scribe Online, Scribe Software Corp.

All About The Cloud - Wall Street Journal, Politico, Bloomberg Panel

This week, we’re in San Francisco at the All About the Cloud Conference, where independent software vendors (ISVs), resellers, platform and infrastructure providers come together with analysts, venture capitalists and the media to talk about what’s hot in cloud computing. For the seventh year, the Software and Information Industry Association (SIIA) is sponsoring the conference to give the industry a place “explore, debate and discuss” the evolving way users consume IT services and access applications. 

The All About the Cloud Conference begins just as MetraTech is expanding its capabilities around the world, particularly in Asia-Pacific. Gartner predicts that region is becoming more and more all about the cloud. The research firm says software-as-a-service (SaaS) revenue in Asia-Pacific countries should reach $934.1 million in 2012, up from the $730.9 million mark it had in 2011. Businesses within Asia-Pacific and around the world are adopting cloud-based delivery models. As they do so, they are faced with the impact of that shift on their monetization models.

That’s where Metanga comes in.

At All About the Cloud, we’ll “explore, debate and discuss” trends in cloud computing with the ISVs who are on the front lines of this evolution. Want to meet up at the conference? Contact us at craffi (at) metratech.com. 

It’s been a big week for SaaS in the Asia-Pacific region. Earlier this week, MetraTech announced a new data center in the region to support Metanga’s growing customer base. We also shared news of our newest customer, PT Infinys System Indonesia. The company chose Metanga as the billing and monetization partner best suited to help grow its business across Asia.

Gartner recently looked at the state of software-as-a-service in Asia-Pacific and found evidence of rapid and significant growth. According to the research firm, the region will best its 2011 SaaS revenue of $730.9 million in 2012. Asia-Pacific, says Gartner, is on pace to capture $934.1 million in SaaS revenue this year.

Our customer PT Infinys System is part of that boom. Established in 2007, PT Infinys positioned itself as Indonesia’s first hosted exchange provider and has since evolved into a full public cloud provider, offering SaaS services as well as infrastructure-as-a-service (IaaS) and platform-as-a-service (PaaS). Metanga helps PT Infinys System monetize those offerings with consumptive metered models that make the business more flexible and more profitable.

In announcing the news of our deal with PT Infinys System, the company’s general manager, Dondy Bappedyanto, laid out the value of Metanga’s monetization platform. He said, “In addition to providing us the tools to maximize our recurring, subscription and metered revenue, Metanga has proven to be an incredibly cost-effective service. Partnering with Metanga was an important first step in our goal to lead the market for cloud services in this region of the world.”

In an area where cloud services are big and growing rapidly, that’s an ambitious pursuit – one fueled by a monetization and billing platform built specifically for SaaS.

In the midst of a monetization landscape working to incorporate dynamic pricing, virtual currencies, mobile banking and other innovations driven by customer behavior, payment professionals will gather next week to talk about the state of the industry. When NACHA, the electronic payments association, meets in Baltimore April 29 to May 2, MetraTech will be there as well, talking with the expected 2,500 attendees about the technology that will support next-generation monetization and billing models.

NACHA’s Payments 2012 conference will focus on several pressing industry topics, including the changing payments environment. That’s a subject we know well, having worked with numerous independent software vendors (ISVs) to help them manage all aspects of software-as-a-service (SaaS) monetization: customer enrollment, product catalogs, pricing, promotions, invoicing, customer self-care and payment capture.

In addition to these SaaS monetization capabilities, we see more companies considering or adopting virtual currencies to create new revenue streams. Several market reports estimate virtual commerce as a $2.2 billion industry that is likely to grow to $6 billion by 2017. The growing number of social apps and games that feature virtual economies are fueling the surge in demand for billing and compensation software that can monetize those creations, generating as much as 100 times the revenues possible through traditional banner ads.

We’ll be talking with folks at the conference about these issues and others facing the industry. Are you going to Payments 2012 in Baltimore next week? Visit MetraTech in booth #517. 

As cloud-based service providers deliver options to customers, they are beginning to collect valuable data that will form the foundation of better revenue management. In turn, systems that can assist in optimizing revenue are the esecential tools for future CFOs. However, that big data needs to be collected and today the only repository that truly understands the actual revenue position of every customer is the billing system. All the granular transaction data SaaS companies log today will power tomorrow’s predictive analytics processes, leading to more informed, more lucrative business models.

Let’s say company X has been serving customer Y for two years. During that time, the customer has upgraded its product package and added optional product choices based on their growth. What can company X learn from this behavior to increase the lifetime value of customer Y and others in similar affinity groups like it? By incorporating an analytics layer into a dynamic billing solution, company X can answer that question and many others. This is the power of predictive analytics.

Earlier this month, SearchCIO looked at the practice of predictive analytics and found two key trends:

The TechTarget publication used the results of a Ventana Research study to illustrate the potential of this data. The study found that only 13 percent of the businesses it surveyed use predictive analysis, but of those who did, 80 percent ranked it as “important or very important.” Perhaps that’s because the enterprises that have mastered predictive analytics use it to enhance revenue-generating activities: forecasting and marketing (72 percent), marketing analyses (70 percent), customer service or support (45 percent), and product recommendations (43 percent).

Billing is growing up to take its rightful place beside other business intelligence functions within SaaS enterprises. In the coming days and months, we’ll be talking more about predictive analytics and the role we expect it to play in the evolution of SaaS business models and monetization. Stay tuned.

Perhaps you built a version of the Millennium Falcon out of Legos as a kid. It was amazing that all those premade building blocks added up into something amazing.  Similarly today, software-as-a-service (SaaS) lets enterprises knit a lot of different applications together, buying services as provided, customizing them in-house, or creating their own on-premise versions. When is it best to take a Lego-like approach, and when should companies purchase the assembled tools they need? The answer depends on whom you ask.

Some enterprises hold tightly to ideals they formed when most software was on-premise. By building their own applications purchasing software to host on their own servers, these companies expect they will save money, increase security, and retain enough control over their tools to adjust functionality as needed. All of that sounds great, but the truth is, unfortunately, sometimes quite different.

After investing the time, human resources and capital in developing or deploying their own solutions, many businesses find themselves behind on their savings expectations. The same, unfortunately, is often true of security. While corporate IT might feel safer hosting its own applications, that feeling can be misguided. When security is breached, the limited internal team might chase the source of the problem fruitlessly for hours or days, whereas the SaaS provider is more likely to detect early warning signs and be prepared to head off trouble before it occurs – all in a faster, more efficient manner than the enterprise can manage itself.

Slowly but surely, more businesses are realizing that custom-built applications can be a painful headache, and SaaS delivers a reliable cure. In 2011, Forrester Research noted that a group of surveyed companies were coming to this conclusion. The “Balancing Custom and Packaged Apps in Your Application Portfolio Strategy” report noted that companies have a growing number of SaaS-based applications in their portfolios. The move toward SaaS, said Forrester, comes from end user demand for highly specific capabilities.

Once, meeting those demands would have required resource-intensive customization of on-premise software or building proprietary solutions. SaaS gives enterprises the means to avoid those onerous tasks while serving end users better and more cost-effectively than was ever possible before.

One of the final questions we asked enterprise users last fall when we polled them about their cloud service and monetization needs had to do with charge back capabilities. The responses were variable, to say the least. Perhaps the one solid takeaway from their feedback is this: cloud providers need to stay flexible and responsive to individual charge back expectations.

From more than 20 enterprise respondents, we’ve culled this list of five potential charge-back challenges cloud providers could face:

1. The term “charge back” itself is up for discussion. There was a wide divergence of how people defined charge-back and how they managed it. (For our purposes, “charge-back” refers to enterprise customers allocating specific costs associated with the use of cloud services and charging those costs back to the organizations for services that were requested and used.) 

2. Larger enterprises want to handle their own charge-back activity. The respondents from larger enterprises said they prefer to maintain their own internal processes and systems rather than having a cloud service provider implement chargeback-as-a-service. Smaller and mid-sized customers reported more inclination toward chargeback-as-a-service subscriptions. 

3. Users rely on their billing systems to calculate charge-backs. This was the overwhelming practice, but some enterprises indicated that they send the resource utilization and charge information from the billing system into a business intelligence system, which then performs the charge-back calculations. A few said they might consider using an external service (such as a SaaS charge-back service).

4. There is no dominant model for cost allocation. Enterprises evaluate the use of cloud resources and services in various ways. One enterprise customer reported charging on a what-you-asked-for basis instead of by a what-you-actually-used method. This approach can be difficult to compute, especially in the case of shared resources. Others used historical utilization as a metric to determine how to distribute costs, while some used headcount, looking at how many members of a department used a particular service or resource.

5. Enterprises expect detailed billing data from their cloud services providers. In general, we found that customers expected the cloud services provider to share detailed billing information on resources used, timing, quantity, and other metrics on a per-account, per-user basis.