Prasenjit Bhadra, CTO – Cygnus Professionals Inc
The evolution of Big Data and Advance analytics in enterprise IT ecosystem has caused a widespread disruption in business transformation initiatives across all verticals. The initial success in managing and manipulating large volume of complex unstructured data in social media and some of the compelling internet products of 21st century, the progressive organizations have gradually started unlocking the potentials of Big data and advance analytics solutions to capture and aggregate large volume of data to aid strategic decision making, fostering innovation and, diversifying the role of IT in complementing product and service offerings. Curious IT leaders are actively evaluating various products and platforms to aid the strategic initiatives around Big Data and Advance Analytics solutions in order to extend actionable insights from a large volume of information and support critical business imperatives. Some of these emerging trends are plugging new dimensions to the enterprise architectural strategies and IT transformation initiatives in terms of addressing the mandates of deriving business values out of discrete information hubs, establishing long-term ROI proof points as well as gaining competitive advantages. The upcoming trends of big data adoption and stretching advance analytics across the value chain shifting enterprise architects focus from common standards of design implementation and governance of IT systems to meet the new challenges of information explosion, changing behavior of the business technology services and, adhering to products/ solutions to maximize the business outcome.
Some of the quick wins with big data analytics have inspired organizations to expand the footprint of contemporary DW&BI foundations, Real-time Analytics and intelligent service automations using large scale distributed data and rapid processing. The proliferation of big data has definitely helped organizations to realize big ideas but has remain myopic on the big picture of long-term strategies and potential impacts on enterprise IT ecosystems.
With current state of maturity, most of the big data centric innovations have taken in LOB specific niche business drivers to craft siloed solutions. The IT leaders, evangelists and Enterprise architects are mostly housing solutions by assessing the readiness of the organization, tools and middleware or products fits the immediate needs and often, developing of reference architectures isolating AS-IS state of the enterprise architecture. Some of the critical impact areas as mentioned below, are seeking due attention of the enterprise architects in order to articulate forward looking strategies and minimize the unforeseen impacts of big data adoption. As we have started to believe that the implementation of big data and advance analytics is an enterprise-wide initiative, we revisit the semantics of process, infrastructure and solution architectures along with new breed of information architecture principles influenced by big data and advance analytics best practices.
Some of our experience and realizations will help EA practitioners build a fresh outlooks assimilating Big Data and Advance Analytics innovation strategies to develop a holistic future state architecture.
Alignment of Business Architecture with Operational Intelligence demands an information driven execution. Traditional business systems, ERP and vertical solutions have emphasized on process centric automation where data are carried over and processed through a series of tasks. The deployment of big data and advance analytics indulges proactive computing or prescriptive and predictive analysis which often get injected within existing operational environment. A new solution within the existing portfolio would also require service and/ or information level integration to support an extensible business intelligence foundations. EA practitioners have to deploy a robust segment architecture methodology to evaluate the current and future impacts and identify the touch points of the process and integration architecture. The process of analysis and provisioning Big Data Analytics should maintain an absolute transparency of the business architecture and provision the subsystems to minimize the impact of the changing application and infrastructure portfolio. EA practitioners have to also assess each application requirements and information architecture against standard matrices of volume, velocity, Variety, Variability, value and, Veracity to qualify the solution as a candidate for big data analytics…
Rethinking Information Management Strategies shift focus from slim and compressed persistence store to a large scale distributed data aggregation. The traditional approach of data management, DB clustering/ partitioning, Enterprise Information Integration/ MDM, Data warehousing principles have least relevance to the emerging standards of Big Data implementation architectures. Any conscious attempt of normalization, replication, archiving and purging data sets are no longer relevant to the data architecture and DBA responsibility. The stakeholders would rely more on the appropriate choice of tools, framework, Meta data designs and thoughtful insights of the data scientists to smoothen the data sets and eliminate the ‘dark data. An ideal solution for a big data and advance analytics would delegate the data management activities on preconfigured processes and built-in features of the platforms. Along the same line the EA practitioners should focus on robust strategies to build a hybrid architecture where the traditional RDBMS/ DW or OLTP databases coexist with big data infrastructure. The information governance policies should also clearly layout the approaches to handle internal and external source of information, B2B integration and defining data islands for common usage. Most of the advance analytics solutions aids competitive business strategies and innovation imperatives. As the footprint of Big Data and Advance Analytics grows considerably, the stake holders have to factor in the security and privacy of the data within the governance policies. While the investments on resources and support services around managing Big Data infrastructure becomes a major concern for most of the enterprises, the ‘Quality of curtailment’ of the expanding data sets will remain a critical decision point for EA practitioners. Therefore, creating fit-to-the-purpose data hubs with relevant data management strategies ensure scalability and extensibility of big data and advance analytics solutions.
Leveraging platforms oriented design to abstract the processing and integration from the core data management. And, this is a Big Step for an effective Big Data adoption! While enterprise continues to invent on new platforms, infrastructures, middleware to shape big data and advance analytics strategies, the implementation of pattern detection, data staging, reusable assets for data smoothening, computational models and visualization frameworks should evolve as a unified platform for evolving application needs. Such platforms can be built by aggregating bespoke tools and scratch built modules which represent the structural and behavioral models for services, process and integration foundations. The approach helps EA teams to refine the reference architecture for Big Data and advance analytics implementations by demarcating, ‘persistence-processing-presentation’ architectural metaphors. Mapping the expectation of speed, performance and quality of the outcome, the platform would embed a distributed intelligence within the system to determine the level of proximity of computational process to the persistence and business service layers. Such platforms will bring strong domain relevance to the implementation strategies and, formalize skill and resource alignment strategies for the IT and business teams.
While organizations focusing on unexplored strategic solutions on big data and advance analytics to complement business IT transformation initiatives, the spur of advance analytics and data intense decision modelling will be assimilated within mainstream business systems gradually. The role of Enterprise architects will not be restricted to define the process, standards and architecture of the enterprise scale big data infrastructure and thereby, realizing disruptive innovations. The proliferation of new tools, applications, platforms and Middleware are elevating the role of IT leaders to develop a data savvy business culture across the value chain, develop measurement standards’ to evaluate the importance of the data and intelligence against the projected total cost of ownership and most importantly, play an active role in suggesting potential business values, insights and opportunities from growing volume of structured and unstructured data.