10.3.1 · A System developed and utilised in products, services, or other use cases that…
10.3.1 A System developed and utilised in products, services, or other use cases that may impact a Data Subject,
negatively or positively, must be designed in accordance with the following concepts:9
(a) Ethical: algorithmic decisions and the associated data lineage of a System should be unbiased and
mitigated. This principle is closely linked with the principles of fairness and transparency.
(b) Fairness: Systems should be designed to treat all individuals equally and fairly, regardless of race,
gender, or other specifically subjective factors. Additionally, Systems should be designed to avoid
potential biases, including unjust bias, or where possible, mitigate bias that could lead to unfair
outcomes.
(c) Transparent: a System must ensure that Processing of Personal Data is explainable to Data
Subjects and other stakeholders in non-technical terms, with appropriate supporting evidence.
(d) Secure: a System must keep Personal Data protected and kept confidential and prevent data
breaches which could cause reputational, psychological, financial, professional, physical or other
types of harm.
(e) Accountability: a System must have mechanisms in place to ensure responsibility and
accountability for enabling its Systems and outcomes. Such mechanisms may include internal
governance and control frameworks in place for monitoring the System, processes and projects
regularly or external organisation auditing processes regularly, enabling the assessment of
algorithms, data and design processes.
9 Guidance on Regulation 10.3.1(a) – (e): This provision gives force to the fundamental principles of fairness, ethical compliance, transparency,
security of operation and accountability that each System that is used to process Personal Data must ultimately comply with. The regulatory
purpose of the provision is to establish these principles as overarching requirements that Systems must be designed to comply with. However,
the provision purposefully does not limit its application to “developers” of Systems only, because it anticipates that ultimately it may be
“Deployers” and “Operators” that, being the visible entities with direct exposure to any data subjects impacted by the operation of the System,
must be held accountable for compliance. “Deployers” and “Operators” would then ensure in turn that they procure Systems only from
“developers” that can give them contractual comfort of compliance-by-design with these principles.
DATA PROTECTION REGULATIONS
13
CONFIDENTIAL
(e)(f) Safety: a System must be designed, developed, and operated with safeguards to identify the use of
personal data, assess and mitigate risks of harm to individuals, and intended to prevent
discriminatory or biased outcomes through appropriate oversight, testing, and controls.