JP Morgan Chase's Hadoop system. Has the time arrived to review Big Data analytics infrastructure?

 

JP Morgan Chase's Hadoop system. Has the time arrived to review Big Data analytics infrastructure?

Dr. Prasad Kulkarni- Consultant- Britts Imperial University Sharjha.

 

Hadoop

 

It's an open source platform used to solve big data using a network of computers. It provides a framework for distributed storage and processing of big data using the Mapreduce programming model.  It is built on the principles of framework should handle the common hardware problems.  Hadoop is versatile and can be used for different tasks. More than this Hadoop offers scalability to enterprises.

Hadoop in the financial Sector

Risk Modeling

Financial sector faces the problem of credit risk, market portfolio risk, and operational risks.credit risks are subject to the bankruptcy of debtors whereas market portfolio risks arise due to inverse returns in portfolio returns. Further, operational risks emerge due to failure of organization’s processes[1]. These risks made financial sector enterprises maintain multiple databases. However, for analysis assembly of files into a  single repository is necessary. This resulted in financial sector companies trusting on Hadoop.

Mass customization:

Financial sector is becoming competitive. To be in the forefront, financial services firms are personalizing their offers. A few firms took a step ahead to offer customized products to clients. This mass customization effort coupled with risk analysis required a quality repository that Hadoop can support.

 

Local data warehouse v/s cloud data warehouse

The year 2008, is the turning point in the financial sector. Till then banks were over protecting their customer data in the local data warehouse. It resulted in the financial crisis as many had no idea about customers other than data submitted to banks. These financial institutions began unearthing the information from emails sent by costumes, their call center conversations, and chat sessions with company representatives. The data produced was enormous for financial services firms. A few of them adopted Hadoop Mapreduce for sentiment analysis, text analysis and behavioral analysis using cloud solutions.

Market predictions

A financial firm has to keep a tab on the stock exchanges how the company and its competitors are performing. It also keeps vigil on regulatory bodies and their change in policies that may affect the firm in the future. The challenge was these data sources were independent and needed integration. Hadoop has worked on integration of these sources to provide more clear insight about the market for financial services firms.

About J.P.Morgan chase

J.P morgan Chase is the largest bank in the USA, headquartered at NewYork city. The bank is named in the Fortune 500 list at the 24th position. It provides investment banking and financial services[2] The former chemical bank merged with Chase Manhattan  corporation in 2000 and was renamed as the J.P. Morgan Chase. JPMorgan's business consists of four main segments: Consumer and Community Banking, Corporate and Investment Banking, Commercial Banking and Asset Management. J.P Morgan Chase built on principles of providing exceptional customer service with integrity and responsibility. J.P.Morgan Chase is a leader in investment banking, financial services for consumers and small business, commercial banking, financial transactions processing and asset management[3].

J.P.Morgan Chase Hadoop system

Credit card information:

J.P.Morgan Chase collects huge data from credit cards of customers. The unstructured data is supported by the Apache Hadoop framework.

Customer service information

The company gets a large number of emails from customers . Though it puts in a relational database but started using an open source framework. It helps the company to do proper risk management[4]

Challenges to Hadoop in 2022.

Emergences of new technologies

Hadoop which was a replacement for relational databases is facing stiff competition from Spark which is internal memory based.  Further, AWS and Microsoft Azure provide cloud based service with faulty tolerant distributed computation at affordable price.

Smile file problem

Hadoop was developed for large files. As we are entering the world of specialized niche software, files may be smaller but Hadoop Mapreduce can not handle data less than 128 MB[5].

Real Time analytics

Hadoop works on batch processing. Hence it is very slow in processing. Developers are mounting Spark on Hadoop systems to get real time analysis[6].

Path ahead for J.P Morgan Chase.

J.P. Morgan Chase  began using Sqrrl, a market app collecte tax saving data, SIP mutual funds and goal based investments for big data analytics. It integrates different datasets and offers data security. The app works on graph analytics to find any outliers in the data security[7].

J.P morgan availed the services of Palantir . This big data analytics software integrates unstructured and structured data to improve the search capabilities. Further, the software helps J.P. morgan chase to to integrate the data and qualitative analytics. This helped the company to identify the  internal fraud in the company[8].

Datawatch another application in big data analytics leveraged by J.P.Morgan Chase to predict market information. It developed on the Data watch platform to keep tabs on real time data arising on the web pertaining to domains of J.P Morgan chase. And offer solutions to the company.

The adoption of niche applications  and emerging trends in Big data technology raised following questions to J.P. Morgan

1.      To continue or not with the Hadoop system for the future?

2.      Is there a necessity for mourning Spark on Hadoop for faster and real time analytics?

3.      Does the company continue to use niche software for specialized functions and create data platforms or look out for third party vendors offering end to end solutions?xctionaclient service; acting with integrity and responsibility; and supporting the growth of our employees.

eliv

References:

1.      Risk Modeling. (2022). Retrieved 11 October 2022, from https://nms.kcl.ac.uk/reimer.kuehn/riskmodeling.html

2.      JPMorgan Chase - Wikipedia. (2022). Retrieved 11 October 2022, from https://en.wikipedia.org/wiki/JPMorgan_Chase

3.      Our Business. (2022). Retrieved 11 October 2022, from https://www.jpmorganchase.com/about/our-business#:~:text=We%20are%20a%20leader%20in,transactions%20processing%20and%20asset%20management.

4.      How JPMorgan uses Hadoop to leverage Big Data Analytics?. (2022). Retrieved 11 October 2022, from https://www.projectpro.io/article/how-jpmorgan-uses-hadoop-to-leverage-big-data-analytics/142

5.      The Good and the Bad of Hadoop Big Data Framework. (2022). Retrieved 11 October 2022, from https://www.altexsoft.com/blog/hadoop-pros-cons/

6.      Chaturvedi, V. (2014). When to and when not to use Hadoop. Retrieved 11 October 2022, from https://www.edureka.co/blog/5-reasons-when-to-use-and-not-to-use-hadoop/

7.      Bajaj, K. (2018). Sqrrl: This free app will solve all investment related confusion. Retrieved 12 October 2022, from https://economictimes.indiatimes.com/magazines/panache/sqrrl-this-free-app-will-solve-all-investment-related-confusion/articleshow/62862969.cms?from=mdr

8.      Top 6 Big Data and Business Analytics Companies to Work For in 2022. (2022). Retrieved 12 October 2022, from https://www.projectpro.io/article/top-6-big-data-and-business-analytics-companies-to-work-for-in-2015/107