Will Regulators Mandate Multi-Region over Multi-AZ in Cloud ?

Multi AZ datacenters

There was a time when one could simply point to the a,b,c of the three data centres in a Multi-AZ (Availability Zone) configuration as a satisfactory replacement of traditional Data Center–Disaster Recovery (DC-DR) requirements even for the regulated sectors. However, as cloud providers face more frequent outages, it’s becoming harder to consider this setup a reliable solution.

Just recently, Google Cloud (GCP) experienced a significant outage lasting about 12 hours, which took down the entire europe-west3 region in Frankfurt, Germany. The root cause was traced to a power and cooling failure, which forced a portion of a zone offline, affecting a wide range of services from Compute to Storage.

Traditionally, disaster recovery setups have involved multiple data centers designated as Primary and Secondary (or DR centers) with data replication. Often, these DC and DR are in separate cities. In the companies that I worked, these were atleast 300 kms apart. While effective, this approach is costly and challenging to maintain, with lot of administrative overhead.

This raises questions about how fintechs and other regulated sectors meet Business Continuity and Disaster Recovery (BCPDR) requirements in the cloud.

In cloud environments, redundancy can be implemented in several ways, with Multi-AZ setups being one of the simplest. Multi-AZ configurations typically involve three data centers within a city, usually spread 30-100 kilometers apart and identified as zones “a,” “b,” and “c.” Data is replicated in real-time across these zones, allowing this setup to be marketed as an alternative to traditional DC/DR arrangements. However, Multi-AZ is not a default feature; it requires opting in and incurs extra costs.

Another approach is Multi-Region, where data is replicated across geographically distinct zones, often in different seismic areas. This setup helps mitigate the risk of a single region being impacted by events such as severe flooding, prolonged power outages, political unrest, and similar disruptions.

Interestingly, some financial institutions, especially Banks are not entirely sold on Multi-AZ setups, pushing their technology partners to adopt multi-region architectures across separate seismic zones. While no regulatory body in India has mandated multi-region setups yet, it’s worth considering the distance between zones within each cloud provider:

  • Amazon AWS states that its Availability Zones are up to 60 miles (~100 kilometers) apart. Interestingly, AWS keeps the exact locations confidential, even from most employees. Source: https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/availability-zones.html#:~:text=Availability%20Zones%20in%20a%20Region,with%20single%2Ddigit%20millisecond%20latency
  • Microsoft Azure mentions a minimum of ~400 kilometers between its AZs, which is the greatest distance among major cloud providers. Source:https://learn.microsoft.com/en-us/azure/reliability/cross-region-replication-azure#what-are-paired-regions)).
  • Google Cloud (GCP) does not publicly disclose the distances between its AZs, keeping this detail confidential. surprising!

Given these developments, regulators may eventually require multi-region setups for regulated entities and fintechs using cloud services instead of relying solely on Multi-AZ. So far, Multi-AZ has offered a relatively straightforward solution to meet compliance and audit requirements for BCPDR. However, it might be time to reconsider RTO (Recovery Time Objective) and RPO (Recovery Point Objective) expectations in cloud environments.

Any thoughts on these developments?

  • Link to the RCA: GCP Outage Incident Report https://status.cloud.google.com/incidents/e3yQSE1ysCGjCVEn2q1h)
  • Article on 12 hour outage: https://www.theregister.com/2024/10/25/google_cloud_frankfurt_outage/

 

When Should Startups Hire a CISO ?

 

When Should Startups hire a CISOAs someone who has been in the IT industry for two decades, primarily working with startups and serving as a security practitioner and consultant now, I often get asked about the right time to bring in full-time security professionals.

These roles could be a Security Engineer, Security Architect or even a more senior role as Chief Information Security Officer (CISO).

But, before that, we need to understand that Startups typically go through various stages of funding and evolution as they grow and develop their business. It is essential to understand how this funding impacts the decision making around security budgets and hires. 

While there’s no one-size-fits-all answer to when startups should hire security professionals. It depends on various factors, whether a startup is in the regulated sector or caters heavily to customers in regulated sectors like Lending, Insurance and Broking or high tech. Does the startup deal with a lot of Personally Identifiable Information (PII), operates across the world, etc.

The reason to put forth this thought is in some sectors, the regulatory requirements are stringent and calls for granular security controls for data protection needing investments in terms of tools, services and people at a much earlier stage. Can startups afford a full time security role at such an early stage? Usually the answer is no and, I think a good alternative lies in the utilisation of Virtual CxOs or Fractional CxOs.

These roles, such as Virtual Chief Information Security Officer, Virtual Chief Technology Officer, Virtual CFO, and others, offer startups invaluable services at a fraction of the huge costs. 

This approach not only provides cost-efficiency but also circumvents the challenges associated with lengthy hiring cycles and the complexities of downsizing. Essentially, these arrangements operate on a ‘pay-as-you-go’ model, offering startups flexibility and strategic support without the traditional hiring headaches. 

So, when can Founders bring in the virtual or a full time CISOs ? Before you go there, a word about CISOs:

CISOs come in all flavours! Some are compliance oriented and usually have transitioned into vCISO roles after a long stint in large enterprises. They would be supported by a team and may be less hands-on. On the other hand, you have the hands-on CISOs, the ‘get your hands dirty in the code’ kind of experts. They’ve seen the trenches, fought the cyber battles, and they’re not afraid to roll up their sleeves.

Both types bring immense value, but choosing the right flavour depends on your startups needs and culture.

Here are the common stages in the funding or evolution of startups and what expectations are from the security roles, be it virtual or full time. Hope you find them useful and please do reach out if you need any help (you can mail me: prasanna@cyberquotient.in):

Pre-Seed Stage:

  • This is the earliest stage of a startup, where the founders use their own funds or money from friends and family to validate their business idea.
  • Funding Source: Founders, friends, and family.
  • Security expectations / deliverables: Since validation of ideas is the prime focus here, a full-time security role is obviously overkill. At the most, the founders would want to know regulatory cyber security compliances of their domains, high level security architecture reviews, security of tech stacks etc. If this is the case, you can talk to  a vCISO or a trusted security professional from your family, friends circle or professional network. 

Early-Stage / Series A:

  • At this point, the startup has a validated product or service and is looking to scale its operations. Funding is used for market expansion, team building, and further product development. 
  • Funding source: Venture capital firms, Angel investors, 
  • Security expectations / deliverables: A minimum viable security of the products,  infrastructure, hygienic data security and privacy controls, High level architecture reviews, are the expectations. Also, specific compliances related to data security and privacy, etc., are to be baked in products and services offered. 
  • Which CISO to hire? A full time CISO / Security Engineer would be an overkill and Startups can benefit from Virtual CISO, who has a mix of Tech and compliance skills, with practical experience. Some of these tasks can be outsourced and accomplished through partner ecosystems. Having a dedicated security team might find the ever changing priorities of a startup too much to handle. Please do read above on various types of CISOs available. 

Growth Stage / Series B and C:

  • Startups at the growth stage have proven market traction and are scaling rapidly. Funding is used for market dominance, scaling operations, and expanding the customer base.
  • Funding Source: Venture capital firms (Series B, C, and subsequent funding rounds).
  • Security deliverables: It gets interesting here. At this stage the products or services offered by startups are already expected to meet certain baseline. Periodic Vulnerability Assessment and Penetration Testing (VAPT) of the web applications, mobile apps, cloud, end user computing are the standard requirements here along with source code reviews. Policies and procedures relating to security, business continuity, ID and access management, data storage, backups are expected to be in place. A common phenomenon I have seen is that startups, which scaled but underplayed the importance of good security hygiene, ended up with a lot of tech debts / security debts and causing friction among teams. Most teams that bear the brunt of this or affected would be Security <-> Engineering <-> DevOps. Eventually, this affects the scalability and agility. of the startup
  • Which CISO to hire? Hire a strong security leader (it could be CISO / Dir of Security / Head of Security, etc) and ensure they are supported by a team including at least one application security engineer, IT / IT Security engineer, compliance analyst etc.

Late-Stage / Series D and Beyond:

  • These startups are well-established in the market and may be preparing for an initial public offering (IPO) or considering other exit strategies. Funding is used for global expansion, acquisitions, and preparing for exit.
  • Funding Source: Venture capital firms, private equity, and sometimes institutional investors.
  • Security deliverables: Same as Series B & C, but the frequency and customer demands, or compliance requirements are more. Periodic VAPT of the web applications, mobile apps, cloud, end user computing. Internal and External audits may be a mandatory requirement. 
  • Whom to Hire: Hire a capable, hands-on security leader (it could be CISO / Dir of Security / Head of Security, etc) and ensure they are supported by a team including at least one application security engineer, IT / IT Security engineer, compliance analyst etc. This stage might also have similar challenges of tech / security debts for the incumbent.

IPO (Initial Public Offering):

  • Some startups choose to go public, offering shares on the stock market. This provides liquidity for early investors and allows the company to raise significant capital from the public.
  • Funding Source: Public investors who buy shares on the stock market.
  • Security deliverables:  At this stage, the organisation is expected to meet higher standards of security including certifications to standards like ISO 27001, although desirable, its not a must. Usually a third-party conducts readiness assessments covering various aspects such as information security,  cyber resiliency, and more. One can expect to see lot of documentation work from the security team including threats and disruptions that can impact the business. The vCISO / CISO will contribute towards the Draft Red-Herring Prospectus (DRHP) documentation as well (I tell this from my experience at Navi).
  • Whom to Hire: Contrary to popular belief, a CISO position is not mandatory for filing IPO, unless you are in regulated sector like Banking, Insurance or Mutual funds. At this stage, a company is expected to have a steady state of security practice. However, startups that grow fast and scale exponentially often face challenges in their security operations. I have seen some startups facing continuous issues and disruptions when tech/security debts have piled up! Look for someone with prior experience in building and scaling security programs at startups.

It’s crucial to understand that not every startup adheres to this precise progression, and the path to funding can fluctuate depending on factors such as industry, business model, and market dynamics.

Similarly, the approach to security may also vary. The consequences of non-compliance can be significant, as illustrated by a recent incident where the RBI imposed hefty fines, viewed by many as potentially fatal.

The era where examples like Uber’s rapid growth leading to regulatory adjustments served as a common case study seems to have shifted. Instead, it’s becoming more evident that it’s a matter of:

‘Comply or Perish’.

 

 

How A Search For “Free Netflix Trial” Led Me To Uncover A Fintech Fraud

 

fintech compliance, RBI Notice #paytm #compliance #fintech

 

This article reminds me of an incident from my past life. I used to work for a Fintech company that was also into PrePaid Instrument wallets, powering one of the biggest utility service providers in India.

One day while doing an Attack Surface Management exercise, something strange happened. I noticed that our company name was quite popular on Youtube and associated hashtags were “free Netflix trial”, #freenetflix. This was around the time Netflix had entered India.

Attack Surface Management (ASM) is basically trying to find out all the references and entry points into an organisation’s servers / apps/ IPs on the web. This is done by doing deep searches using google dorks, dark web/deep web in both manual and automated ways. Nowadays, there are third-party companies that specialise in these.

Trivia: Interestingly, Cyber Insurance companies also use these ASM techniques to understand more about their clients digital footprint, exposure etc., and use it for calculating Cyber Insurance Premiums.

Curious, I dug deeper and found out a youtube video where an influencer had put up a tutorial on how one could easily create a virtual prepaid card in our platform, loading it for INR 2/- and then use that for Netflix trial for about a month and then recycle the whole process. I figured out that:

They were using the same Driving License (DL) numbers and sometimes recycling DL numbers by appending a digit. Since, in Minimum KYCs only numbers and some basic details like name and etc were collected, usually without any kind of photocopy of the documents. I believe this process has improved now (?).

I took those details and with the help of the internal teams, started researching more about those DL numbers. Turns out there were close to a few hundred accounts associated with that single number. Ideally, the systems should have flagged and blocked subsequent wallet creation using the same ID numbers. Somehow it was failing.

rbi non compliance

We looped in Compliance, FRM and legal. Such accounts with dubious credentials were invalidated and we started rolling out additional controls.

It was trivial to use the same numbers, unless there was a robust mechanism that checked the Document ID numbers and the associated identifiers like Mobile number, names etc.

Though I was in cybersecurity, this was my first brush with fighting digital fraud.

In another incident, fraudsters had cloned our fintech app and we had to loop in our legal teams to send a Trademark infringement to the AppStore to get this fake loan app removed.

In security we analyse the Tactics, Techniques and Procedures (TTPs) that are used commonly by the hackers. This helps us security teams in detecting and mitigating attacks by understanding the way threat actors operate.

Adapting a similar approach of examining the tactics, techniques, and procedures used by fraudsters will provide valuable insights into their behaviour and motives. With this understanding, the Fintechs and FIIs can help develop effective countermeasures. A good start would be documenting all the counter-fraud use-cases !

A key takeaway or a piece of advice to all the fintechs is to build a comprehensive list of use-cases proactively, and have them tested thoroughly. You can take help from your InfoSec teams in building these use-cases!

What do you say ?

#DigitalFraudPrevention #fintech #fintechfraud #search #attacksurface #TTPs #Tactics #Techniques #Procedures #usecase #ThreatActorAnalysis #frauddetection

Fighting Fraud Loan Apps with PKI

Fintech loan app scams

Rise of Fraudulent Loan Apps During Covid

In the aftermath of the Covid-19 pandemic from 2020 and its subsequent decline, there has been a notable surge in fraud cases, particularly revolving around counterfeit fintech apps offering loans. Individuals unsuspectingly downloading these deceptive loan apps are not only falling victim to steep interest rates but also compromising personal data such as photos and contacts. There have been following media reports where some have even committed suicide:

News report on Fake Loan Apps

This has escalated into a relentless cat-and-mouse game, with the odds heavily favouring fraudsters. I have closely followed some great work done by folks like Babu Lal in getting these fake loan apps removed from the playstores:

Babu Lal fighting against fake loan apps

Regulatory bodies are deflecting responsibility onto Play Stores, but I think that addressing this issue requires a collaborative effort between the Reserve Bank of India (RBI), Non-Banking Financial Corporations (NBFCs), and Play Stores operated by industry giants like Google and Apple.

Play Stores employ a gating check for allowing fintech apps, which involves verifying the NBFC certificate issued by the RBI. This certificate is essentially a copy of the physical document provided by the RBI to NBFCs/Banks/Financial Instituitions.

What has come to my attention, through both extensive reading of various reports and first-hand encounters with similar incidents, is that:

Fraudsters are Fabricating copies of the RBI-issued NBFC certificates.

These fabricated copies are then submitted to Play Stores along with the mobile app applications. It’s essential to note that sharing this NBFC certificate is mandatory for apps to be categorised as financial apps.

In the case we encountered, it took a significant amount of time to get the fake app removed from the playstore. This was an OEM playstore and not the regular Google or Apple owned.  The resolution ultimately necessitated pursuing the trademark infringement route.

The Solution:

One of the ways I thought this could be curbed or to an extent limited is by making use of Public Key Infrastructure (PKI) setup with Digitally Signed Certificates + Encryption and using the good old PGP/GPG tools. PGP stands for Pretty Good Privacy and originally developed by Phil Zimmermann in 1991 as a proprietary software. While GNU Privacy Guard (GPG) was developed as an open-source alternative to PGP. GPG is essentially PGP with an open-source licence.Yeah, it’s free and has no impact monetarily on the smaller banks and financial institutions for easy adoption. Only a small learning curve in the beginning on usage of GPG, but it’s better than losing money.

The goal is to establish a robust system that ensures the integrity of certificates and prevents the submission of falsified documents to Play Stores, thereby mitigating financial losses and data leaks.

Using PKI for leveraging a Web of Trust already in place

In the context of multiple stakeholders, this proposed solution aligns with the shared responsibility among Apple, Google, RBI, and NBFCs in addressing the rise of fraudulent fintech apps. Here’s a breakdown of the proposal:

1. Establish Dedicated Email Addresses and Publish Public Keys:

– All involved parties would create dedicated email addresses and publish their public keys into a Key Server. This ensures a secure and standardised communication channel.

2. RBI Digitally Signs Certificates for NBFCs:

– The RBI would be responsible for digitally signing certificates in the names of respective NBFCs, establishing a secure and verifiable authentication process.

3. Optional Encryption of Certificates for NBFCs:

– As an additional layer of security, the RBI could choose to encrypt these certificates with the respective NBFCs’ public keys while transmitting them, preventing potential Man-in-the-Middle (MITM) attacks. The reason to make this optional is, there are situations these certificates are attached when interacting with regulators.

4. NBFCs/Fintechs Decrypt and Use Certificates:

– NBFCs and Fintechs would then decrypt these certificates using their private keys and utilise them for their intended purposes.

5. NBFCs/Fintechs Sign and Encrypt Certificates for Playstores:

– NBFCs and Fintechs, having used the certificates, would digitally sign them and encrypt them with Google’s public keys as part of the Mobile App application form.

6. Playstore Teams Validate and Approve Apps:

– Playstore teams can efficiently validate these digitally signed and encrypted certificates, facilitating a streamlined approval process. Automation could be implemented given the inherent security of digital signatures.

7. Preventing Submission of Fake Documents:

– The proposed system effectively prevents the submission of fraudulent documents, such as copies of physical certificates, by ensuring the authenticity and integrity of digital certificates.

I couldn’t think of any loopholes except for the fact that commercial viability of PGP seems to have taken a severe beating. But there is good old GPG software to the rescue. While there has been similar implementation using Blockchain, the biggest hurdle I anticipate is the adoption with the NBFCs and other Financial Institutions that may not be tech-savvy.

The onus of keeping the RBI provided Certificates will be on the NBFCs and they can be revoked by RBI if in case they are leaked.

This is just a high level idea and can be further customised to handle the technical challenges and as reiterated earlier, this is a:

Shared Responsibility between the Regulator: RBI & Others, Playstore: Google, Apple & other Playstores, and NBFC/Fintechs/FIIs/Banks.

Opening this to scrutiny by peers. Please let me know what you think..

Reserve Bank of India (RBI) Google Apple

#fraudapps #fraudloanapps #loanappscams #loansharks #onlineloan #instantloan #loanpps #fintechfrauds