The swift advancement of Artificial Intelligence (AI) and Machine Learning (ML) technology is stimulating a new era of opportunities across several industries. As organizations explore ways to embrace the transformative power of these emerging innovations, they must do so with particular attention to the Governance, Risk, and Compliance (GRC) challenges AI and ML present across various functions. Addressing these challenges is not a one-size-fits-all scenario. Certain issues are unique to distinct sectors, and will therefore require different strategies and solutions to ensure safe and secure implementation, as well as regulatory compliance.
Governance in AI and ML
AI and ML governance requires attention in three primary areas: autonomy, data quality, and fairness. In general, AI governance establishes who is in charge of the AI system’s oversight, and how much of the organization’s daily life or processes can be influenced by its algorithms. A pathway to good AI governance must provide parameters for acting ethically, assigning responsibility, and defining operational policies.
Ethical Frameworks
Managing AI and ML technologies begins by creating ethical frameworks with clear regulations around issues of transparency, bias, and fairness. To apply AI ethically, AI systems’ decision-making processes should incorporate ethics in a way that is consistent with societal ideals. Fintech companies, for example, manage sensitive financial transactions and information. Upholding moral principles fosters trust among stakeholders, partners, and consumers. Long-term business success can be attributed to attracting and keeping clients that have a good reputation for ethical behavior.
Accountability and Responsibility
Well-defined roles and responsibilities are necessary for effective governance. You can ensure that everyone participating in the AI and ML lifecycle, from developers to executives, is aware of their responsibilities by establishing accountability. This duty includes general system behavior, decision-making processes, and algorithmic outputs. Accountability applies to both individuals and groups that create AI systems. They must be responsible for developing and training AI ethically, free of biases from the start, and equipped with safeguards against abuse or mistakes.
Comprehensive Policies
Any organization embarking on AI/ML development must formulate and implement comprehensive policies that cover all aspects of the AI and ML lifecycle, from data collection to model development, deployment, and ongoing oversight. To encourage appropriate AI use, these guidelines should be in accordance with business objectives, industry norms, and legal requirements. Examples of such policies might include:
- establishing duties and responsibilities unique to each employee
- documenting and updating AI policy on a regular basis
- informing stakeholders of standard ethical behavior
- keeping abreast of all applicable laws and regulations.
Risk Management in AI and ML
AI/ML risk management describes a set of procedures and instruments used to proactively shield businesses and end users from the particular hazards associated with AI and ML. This entails calculating the risks, and putting preventive measures in place to lessen the potential impact of any adverse event.
Algorithmic Bias and Fairness
Identifying and reducing algorithmic bias are crucial elements of AI/ML risk management. Conscientious companies will regularly audit their algorithms to ensure fairness for various demographic groups, prevent unintentional discrimination, and address any biases in the training data. During the ML process, an algorithm can be biased by erroneous assumptions and thus produce biased results. In the financial services industry, for example, a biased algorithm may generate unjust loan denials or higher insurance rates for specific demographic groups.
Data Privacy and Security
Data security is one of the primary issues underlying AI and ML applications. Organizations must abide by data privacy rules and have robust security measures to stop unauthorized access and data breaches. An extensive data protection plan needs to have anonymization, encryption, and access limits. One notable instance of a serious data security breach is the JP Morgan event from 2014, in which hackers in Eastern Europe used phishing to gain access to a worker’s personal computer, that then gave them a port of entry to the bank’s network. This incident resulted in the loss of privacy for almost 76 million households and seven million small businesses. Since AI/ML uses a lot of data, data security and privacy need to be top priorities.
Explainability and Transparency
When AI decision-making processes are not transparent, organizations risk grave repercussions. Leadership can mitigate this risk by developing explainable AI models that will aid users and stakeholders in understanding the decision-making process. In addition to building trust, transparent AI assists in identifying and resolving potential issues. For example, banks, credit unions, insurance companies, and other financial service companies must follow certain regulations which dictate how personal data is collected, stored, and processed, and are required to maintain transparency.
Compliance in AI and ML
AI compliance guarantees the ethical and beneficial use of AI-powered technologies for the good of society. At the most fundamental level, compliance demands that no individual or organization violates people’s security or privacy through the use of AI-powered solutions.
Regulatory Adherence
Initiatives involving AI and ML must comply with current legal standards, and keep up with the constantly evolving legal frameworks pertaining to data security, privacy, and the ethical application of AI. To avoid legal repercussions, businesses must understand laws such as the General Data Protection Regulation (GDPR), considered to be the world’s most stringent privacy and security law. Although the GDPR was enacted by the European Union (EU) in May 2018, it can impose obligations and harsh penalties on any organization that collects data on EU residents, regardless of global location. Compliance adherence becomes even more challenging with the use of Generative AI tools such as ChatGPT. For example, in March 2023, Open AI was accused of violating GDPR privacy regulations after a data breach that exposed ChatGPT users’ conversations and payment information.
In the U.S., the Department of Health and Human Services (HHS) included rules for Privacy and Security in the Health Insurance Portability and Accountability Act of 1996 (HIPAA), which established national standards to protect individuals’ health information. This is just one of hundreds of federal and state regulations that govern how data is collected and used. Across all sectors, businesses that fail to respect local, national, and international laws may be subject to severe fines, sinking brand reputations, and/or a loss in enterprise value.
Documentation and Auditing
Organizations must maintain thorough documentation of all AI and ML operations to facilitate audits and regulatory compliance assessments. This documentation should address data sources, model construction methods, validation processes, and ongoing monitoring programs. Regular audits help guarantee that AI systems stay compliant with changing regulatory constraints. Effective auditing practices for AI include setting clear goals, assembling a multidisciplinary team that includes and/or represents all stakeholders, standardizing measurements and tools, performing follow-up audits following major modifications, enlisting the help of third parties, and providing actionable recommendations.
Continuous Monitoring and Adaptation
Due to the dynamic nature of emerging technologies and the related evolving regulatory environment, organizational rules and policies around AI and ML must be constantly monitored and adjusted to stay current and compliant. Businesses should establish procedures for tracking legislative developments and update their AI/ML governance frameworks as needed. This best practice has the added benefit of serving as a preventive strategy to keep AI applications healthy and effective. In the healthcare industry, for example, important conclusions are drawn from the clinical data, but many ML models are highly sensitive in nature and prone to performance decay over time, due to outdated data and model functionality. This can create significant use and  compliance issues. Continuous monitoring can detect and protect against these kinds of system failures.
A Sector-specific Summary
While the GRC guidelines presented above are generally applicable to all organizations employing AI and ML tools, businesses in the following specific sectors should consider issues that may be unique to their industry or services.
- Finance and Banking: Adhering to legal and ethical standards for financial services requires robust governance, particularly when using AI for algorithmic trading, risk assessment, and fraud detection.
- Healthcare: Adopting AI for patient data management, tailored medication, and diagnostics involves carefully navigating ethical and privacy considerations, such as HIPAA rules in the U.S.
- Technology and IT: Organizations that rely extensively on AI, including AI firms, must be especially focused on algorithmic accountability, transparency, and data protection.
- Automotive: Incorporating AI into autonomous vehicles raises specific concerns related to safety, security, and regulatory compliance.
- Retail and E-commerce: Issues regarding consumer privacy, data protection, and fair use of algorithms are raised by the application of AI in customer service, supply chain optimization, and tailored marketing.
- Telecommunications: Compliance with data protection laws and cybersecurity standards is crucial for the telecom industry’s AI-powered network management, customer support, and predictive maintenance.
- Energy and Utilities: The energy industry must handle compliance concerns regarding data security and environmental standards as it uses AI for predictive maintenance, grid optimization, and resource management.
- Government and Public Sector: The use of AI by public entities for a variety of functions, including decision-making, public services, and law enforcement, presents issues with transparency, accountability, and ethical technology use.
- Insurance: Applications of AI in insurance underwriting, claims processing, and risk assessment demand close attention to ethical and industry regulations.
- Manufacturing: When integrating AI into industrial processes, quality control, and predictive maintenance, manufacturers must comply with safety norms, environmental laws, and ethical technology use guidelines.
- Legal and Compliance Services: These service providers must be diligent in monitoring the regulatory environment around AI so as to keep their clients up-to-date and offer correct guidance when moral and legal dilemmas arise.
In any industry, an integrated approach is necessary to effectively manage AI/ML governance, risk, and compliance, and confront  the complicated challenges these tools may pose. Organizations that prioritize ethical issues, implement clear governance structures, manage risks, and ensure regulatory compliance can capitalize on the revolutionary promise of AI and ML, while maintaining trust, transparency, and responsible use of these technologies. Combining AI with GRC principles improves productivity and decision-making in fields such as information security, marketing, and software development. While there is no denying the allure of AI’s powers, the careful control that comes with thoughtful GRC guarantees responsible and moral applications of AI, now and in the future.
Pritam Mukherjee is Lead Specialist for IT Application and Data Security, Governance, Technology Risk and Compliance at the New York Power Authority (NYPA), America’s largest state public power organization. His leadership responsibilities include designing and managing IT application security and GRC in the utility’s digital transformation, integrating SAP Analytics Cloud, and AI/ML, and implementing the utility’s 10-year vision for leveraging advanced technologies and ensuring data security across multiple business processes. Mr. Mukherjee earned a Master’s degree in Financial Innovation and Technology from Smith School of Business, Queens University (Ontario, Canada), and a Master of Technology degree in Computer Engineering from the University of Calcutta (India).