JP Morgan Chase's Hadoop system. Has the time arrived to review Big Data analytics infrastructure?
Dr. Prasad Kulkarni- Consultant- Britts
Imperial University Sharjha.
Hadoop
It's an open source platform used to solve
big data using a network of computers. It provides a framework for distributed
storage and processing of big data using the Mapreduce programming model. It is built on the principles of framework
should handle the common hardware problems.
Hadoop is versatile and can be used for different tasks. More than this
Hadoop offers scalability to enterprises.
Hadoop in the financial Sector
Risk Modeling
Financial sector faces the problem of
credit risk, market portfolio risk, and operational risks.credit risks are
subject to the bankruptcy of debtors whereas market portfolio risks arise due
to inverse returns in portfolio returns. Further, operational risks emerge due
to failure of organization’s processes[1]. These risks made financial sector
enterprises maintain multiple databases. However, for analysis assembly of
files into a single repository is
necessary. This resulted in financial sector companies trusting on Hadoop.
Mass customization:
Financial sector is becoming competitive.
To be in the forefront, financial services firms are personalizing their
offers. A few firms took a step ahead to offer customized products to clients.
This mass customization effort coupled with risk analysis required a quality
repository that Hadoop can support.
Local data warehouse v/s cloud data warehouse
The year 2008, is the turning point in the
financial sector. Till then banks were over protecting their customer data in
the local data warehouse. It resulted in the financial crisis as many had no
idea about customers other than data submitted to banks. These financial
institutions began unearthing the information from emails sent by costumes,
their call center conversations, and chat sessions with company
representatives. The data produced was enormous for financial services firms. A
few of them adopted Hadoop Mapreduce for sentiment analysis, text analysis and
behavioral analysis using cloud solutions.
Market predictions
A financial firm has to keep a tab on the
stock exchanges how the company and its competitors are performing. It also
keeps vigil on regulatory bodies and their change in policies that may affect
the firm in the future. The challenge was these data sources were independent
and needed integration. Hadoop has worked on integration of these sources to
provide more clear insight about the market for financial services firms.
About J.P.Morgan chase
J.P morgan Chase is the largest bank in the
USA, headquartered at NewYork city. The bank is named in the Fortune 500 list
at the 24th position. It provides investment banking and financial services[2]
The former chemical bank merged with Chase Manhattan corporation in 2000 and was renamed as the
J.P. Morgan Chase. JPMorgan's business consists of four main segments: Consumer
and Community Banking, Corporate and Investment Banking, Commercial Banking and
Asset Management. J.P Morgan Chase built on principles of providing exceptional
customer service with integrity and responsibility. J.P.Morgan Chase is a
leader in investment banking, financial services for consumers and small
business, commercial banking, financial transactions processing and asset
management[3].
J.P.Morgan Chase Hadoop system
Credit card information:
J.P.Morgan Chase collects huge data from
credit cards of customers. The unstructured data is supported by the Apache
Hadoop framework.
Customer service information
The company gets a large number of emails
from customers . Though it puts in a relational database but started using an
open source framework. It helps the company to do proper risk management[4]
Challenges to Hadoop in 2022.
Emergences of new technologies
Hadoop which was a replacement for
relational databases is facing stiff competition from Spark which is internal
memory based. Further, AWS and Microsoft
Azure provide cloud based service with faulty tolerant distributed computation
at affordable price.
Smile file problem
Hadoop was developed for large files. As we
are entering the world of specialized niche software, files may be smaller but
Hadoop Mapreduce can not handle data less than 128 MB[5].
Real Time analytics
Hadoop works on batch processing. Hence it
is very slow in processing. Developers are mounting Spark on Hadoop systems to
get real time analysis[6].
Path ahead for J.P Morgan Chase.
J.P. Morgan Chase began using Sqrrl, a market app collecte tax
saving data, SIP mutual funds and goal based investments for big data
analytics. It integrates different datasets and offers data security. The app
works on graph analytics to find any outliers in the data security[7].
J.P morgan availed the services of Palantir
. This big data analytics software integrates unstructured and structured data
to improve the search capabilities. Further, the software helps J.P. morgan
chase to to integrate the data and qualitative analytics. This helped the
company to identify the internal fraud
in the company[8].
Datawatch another application in big data
analytics leveraged by J.P.Morgan Chase to predict market information. It
developed on the Data watch platform to keep tabs on real time data arising on
the web pertaining to domains of J.P Morgan chase. And offer solutions to the
company.
The adoption of niche applications and emerging trends in Big data technology
raised following questions to J.P. Morgan
1. To continue or not with the Hadoop system
for the future?
2. Is there a necessity for mourning Spark on
Hadoop for faster and real time analytics?
3. Does the company continue to use niche
software for specialized functions and create data platforms or look out for
third party vendors offering end to end solutions?xctionaclient service; acting with integrity and
responsibility; and supporting the growth of our employees.
eliv
References:
1. Risk Modeling. (2022). Retrieved 11 October
2022, from https://nms.kcl.ac.uk/reimer.kuehn/riskmodeling.html
2. JPMorgan Chase - Wikipedia. (2022).
Retrieved 11 October 2022, from https://en.wikipedia.org/wiki/JPMorgan_Chase
3. Our Business. (2022). Retrieved 11 October
2022, from https://www.jpmorganchase.com/about/our-business#:~:text=We%20are%20a%20leader%20in,transactions%20processing%20and%20asset%20management.
4. How JPMorgan uses Hadoop to leverage Big
Data Analytics?. (2022). Retrieved 11 October 2022, from https://www.projectpro.io/article/how-jpmorgan-uses-hadoop-to-leverage-big-data-analytics/142
5. The Good and the Bad of Hadoop Big Data
Framework. (2022). Retrieved 11 October 2022, from https://www.altexsoft.com/blog/hadoop-pros-cons/
6. Chaturvedi, V. (2014). When to and when not
to use Hadoop. Retrieved 11 October 2022, from https://www.edureka.co/blog/5-reasons-when-to-use-and-not-to-use-hadoop/
7. Bajaj, K. (2018). Sqrrl: This free app will
solve all investment related confusion. Retrieved 12 October 2022, from https://economictimes.indiatimes.com/magazines/panache/sqrrl-this-free-app-will-solve-all-investment-related-confusion/articleshow/62862969.cms?from=mdr
8. Top 6 Big Data and Business Analytics
Companies to Work For in 2022. (2022). Retrieved 12 October 2022, from
https://www.projectpro.io/article/top-6-big-data-and-business-analytics-companies-to-work-for-in-2015/107
