The European Union’s Artificial Intelligence (AI) regulation establishes a comprehensive framework for the safe and ethical use of AI applications. Regulation 2024/1689 of the European Parliament and Council (AI Regulation), effective from August 1, 2024, aims to ensure that AI development does not jeopardize fundamental rights and the protection of privacy.
The AI Regulation defines clear risk categories, prescribing different levels of oversight and compliance obligations for various AI systems. These systems are categorized into four main risk levels: minimal risk, limited risk, high risk, and unacceptable AI systems.
The AI Regulation allows the unrestricted use of minimal risk AI applications, such as AI-enhanced video games, spam filters, or simple systems that do not pose risks to users’ rights or safety. Most AI systems currently used in the EU fall into this category. However, even these systems must comply with basic obligations, such as informing users when they interact with an AI system, unless this is obvious in the given context.
Limited risk AI systems have minor impacts on individuals’ rights and safety. Examples include chatbots or personalized advertising that do not directly endanger users’ rights but require transparency. The regulation requires that users must be clearly informed when engaging with an AI system. AI-generated content, such as advertisements, must be also be properly identified and labelled. Ensuring transparency helps users to make informed decisions. At the same time, although these systems pose limited risks, there are also privacy and data security risks associated with their use. For example, uploading personal data to such systems could result in data breaches. In the event of a data breach, there are strict compliance requirements, information obligations towards the data protection authority and the data subjects, with potentially serious consequences for the data controller and, of course, for the data subjects. If the uploaded text contains trade secrets, the company could easily find itself in a position of breach of contract with its contractual partners or in a disadvantageous position of compromising its own trade secrets. It should be noted that various Data Loss Prevention tools are capable of detecting if employees are potentially uploading data files to an AI system, such as ChatGPT. However, it is important to remember that these tools also have data protection implications, as they monitor employees’ activities. Employees must be informed if the company uses such tools to prevent data leakage.
The third category is high-risk AI systems. High-risk AI systems directly impact individuals’ lives, rights, and safety. Examples include healthcare diagnostic tools that identify diseases and suggest treatments, credit scoring systems influencing financial opportunities, recruitment tools or performance evaluation systems used in hiring and promotions.
These systems require stringent risk assessments to that users’ rights are protected and that decisions taken by the system comply with the law. The data used in these systems must meet high-quality standards to prevent to minimise discriminatory effects and prevent individuals from being adversely affected by the operation of the algorithms. In the case of high-risk AI systems, human oversight is crucial to mitigate risks, and must be ensured by appropriate control mechanisms. After the introduction of such systems to the market, their oversight becomes part of the responsibilities of the relevant market surveillance authorities. At the same time, users are responsible for human oversight and monitoring the system’s operation. Service providers must ensure the operation of a post-market monitoring system, which enables the effective monitoring of the system’s performance and potential risks.
Data protection considerations are particularly significant for high-risk AI applications, as these systems may process sensitive personal data. The UK’s Information Commissioner’s Office (ICO) highlights the data protection risks and mitigation strategies for AI-based recruitment tools in its report titled “AI Systems in Recruitment.”[1] For such tools, it is crucial to regularly test algorithms for accuracy and bias to prevent negative discrimination. Ensuring transparency is also essential — candidates must be clearly informed about how their data is being used. Additionally, the principle of data minimization must be observed, meaning that only the necessary data should be collected and processed using AI tools. Furthermore, it is essential to conduct Data Privacy Impact Assessments (DPIAs) for AI systems used in recruitment — and, arguably, for any AI system used during employment — to identify and address potential risks in a timely manner. It is important to distinguish between a legal impact assessment required under AI-specific regulations (discussed later) and a data protection impact assessment (DPIA). While the former focuses on the legal and ethical risks of AI systems, the latter aims to identify and mitigate risks related to data processing. Even in cases where AI regulations do not require a legal impact assessment (for instance, when the AI system is not classified as high-risk), it may still be necessary to conduct a DPIA to ensure compliance with the GDPR. It should be noted that while the obligations under the EU AI Act related to high-risk AI systems will only apply from August 2, 2026, organizations using such systems must already comply with the GDPR, which requires DPIAs as part of the broader obligations related to data protection.
The ICO’s report emphasizes the importance of avoiding discrimination when using recruitment tools and ensuring that all data processing aligns with the GDPR principles of purpose limitation and lawfulness.
An illustrative example of these principles can be seen in the decision of the Hungarian National Authority for Data Protection and Freedom of Information (NAIH) concerning Budapest Bank Zrt. (Decision No. 85-3/2022, issued on February 8, 2022[2]). The case revealed how AI use can become high-risk when it processes sensitive data, such as customers’ emotional states. As part of its complaint management process, Budapest Bank used an emotion recognition and analysis system to determine which customers should be called back. NAIH found that the use of this emotion recognition system raised serious data protection concerns, especially in light of the inadequate information provided to customers and the flaws in the bank’s balancing of interests. As a result, NAIH imposed a record fine of HUF 250 million for unlawful data processing.
Following high-risk AI systems, the strictest category is represented by prohibited AI systems. These include systems that attempt to influence users’ decisions or exploit people’s vulnerabilities, such as manipulating individuals with disabilities or members of disadvantaged groups.
Upcoming Deadlines and Compliance Tasks
Regarding the AI system categories mentioned above, the first key date is February 2, 2025, when the rules on prohibited AI systems will come into effect. Companies must assess, before this date, whether any of the AI systems they use do not fall into the prohibited category. This assessment requires an evaluation of the characteristics, functionalities, and risks associated with the AI systems in use. Under the AI Regulation, the use of prohibited systems can result in severe penalties, including fines of up to €35 million or 7% of the company’s global annual turnover.
From August 2, 2026, compliance requirements for high-risk AI systems will be enforced. By this date, all AI systems classified as high-risk must fully comply with the detailed regulatory framework. These requirements include maintaining detailed logs and conducting fundamental legal impact assessments. Non-compliance could lead to fines of up to €15 million or 3% of the company’s global annual revenue.
In addition to the AI Regulation, it is essential not to overlook the obligations under the GDPR.While AI is not explicitly mentioned in the GDPR, Article 22 (automated individual decision-making, including profiling) indirectly applies to the use of AI systems, as these systems often make automated decisions that affect individuals.
To ensure compliance, the first step is to carry out a general impact assessment and risk analysis of the AI systems in use. This process will help to assess the risk classification of the AI systems in use and allow companies to prepare for compliance in a timely manner. Carrying out an impact assessment is particularly important before the February 2025 deadline, by which time prohibited AI systems must be avoided. Such systems are subject to the most severe sanctions under the AI Regulation, so it is essential for companies to review their systems and take appropriate action now.
[1] https://ico.org.uk/action-weve-taken/audits-and-overview-reports/ai-tools-in-recruitment/