Malaysia AI Ethics Maturity Report 2021
Ethical Standards Towards a Trustworthy AI: A Measurement to Determine AI Maturity
Published on 16.11.2021
The content of this report is protected by copyright 2021 AI Doctrina (and individually Assoc Professor Dr. Jaspal Kaur and Darmain Segaran). You may reproduce materials from this report without prior permission for non-commercial purposes on the condition that you provide proper attribution of the source. This page may be cited as source material. Please contact us to obtain permission for any other use. This report may be periodically reviewed for errors however data published is only accurate as at the time of publishing.
1. INTRODUCTION
Whilst promoting the innovation capacity of AI, the development and uptake of ethical and trustworthy AI has become central to the development and deployment of AI. The growth of AI raises several questions and challenges which require careful consideration, particularly when the development and deployment of AI raise ethical dilemmas. The issue of trust is at the forefront of concerns of policy-makers as they attempt to define and frame national policies.
The main concern with AI is a lack of trust. The need to address the issue of lack of trust is essential as it may become an impediment to the use of AI and hence, its ability to promote altruism and utility of AI. Questions of distrust raised surround the risks involved in the development and deployment of AI, which include the lack of transparency in the use and quality of the data or the source of the data used to train the AI, reliance on predictive models of AI used based on biased data or the lack of explainability* or interpretability* of an outcome and decision arrived at by the AI. This distrust raises ethical dilemmas when different forms of AI (neural networks and machine learning algorithms) are deployed in critical sectors such as finance, social care, transportation, healthcare, police enforcement, and the justice process.
With these issues of distrust, comes the determination on the part of policy-makers to make AI a tool for societal good – hence the term “Trustworthy AI” has emerged in the discourse to develop and deploy AI that meets the ethical standard of trustworthiness. An AI Ethics National Framework will assist in guiding individuals, corporations, and organisations to embrace the consideration of ethics in the development and deployment of AI. This is extremely important in Malaysia as the government has adopted a more collaborative approach with the private sector in developing AI capabilities.[1] At the very same time, it is to be noted that a national policy must contribute to the dynamism, innovativeness, and competitiveness of the AI ecosystem.
AI Ethical Frameworks have been drafted at the national, regional, and international levels as well as by the industry. At the regional (e.g. the European Commission) and international level (e.g. IEEE, Asilomar AI Principles, ITI AI Principles), these frameworks have been crafted by organisations that that represent member states or interests, that are then to be adopted at the national or organisational level. These frameworks may vary in terms of the values that are prioritised. At the national level, the lead is taken to draft principles that are adopted and adapted from these frameworks. At the industry level, companies such as Microsoft[2] and Google[3] have developed their respective frameworks. Industry or professional specific frameworks have been established, for instance, in judicial decision-making utilising AI in predictive justice, judges look to the European Commission for the Efficiency of Justice (CEPEJ)’s ‘European Ethical Charter on the Use of Artificial Intelligence in Judicial Systems and their environment’.[4]
Frameworks comprise general overarching principles. As a first example, according to the European Commission Guidelines,[5] Trustworthy AI should be - lawful (respecting all applicable laws and regulations); ethical (respecting ethical principles and values); and, robust (both from a technical perspective while taking into account its social environment). In another example, Microsoft’s Responsible AI[6] refers to - fairness (AI systems should treat all people fairly); inclusiveness (AI systems should empower everyone and engage people); reliability and safety (AI systems should perform reliably and safely); transparency (AI systems should be understandable); privacy and security (AI systems should be secure and respect privacy); and, accountability (AI systems should have algorithmic accountability).
Trustworthy AI is premised on several bedrock principles namely lawfulness, ethical, and robustness. The Survey focuses on the second principle - that is, of an Ethical AI. We identify 5 overarching key assessments which are derived and amalgamated from two sources – firstly, the 7 key assessments found in the European Commission’s The Ethics Guidelines for Trustworthy Artificial Intelligence (AI) which is a document prepared by the High-Level Expert Group on Artificial Intelligence (AI HLEG); and secondly, the 5 value-based principles recommended in the OECD’s Recommendation of the Council on Artificial Intelligence. These 5 key assessments based on value-based principles are transparency and explainability; privacy and data governance; robustness, security and safety; human-centered values, fairness and societal well-being; and accountability. To measure AI Ethics Maturity, the researchers believe these key assessments must be used to evaluate the adoption of self-governance structures that operationalise ethical values.
These key assessments apply to both Developers* and Deployers*of the AI. This has been done with the appreciation that an AI actor may be an organisation that is either a Developer or a Deployer or both. For the definitions of “Developer” and “Deployer”, guidance has been obtained from the Second Edition of the Singapore Model AI Governance Framework 2020 (refer to Glossary of Terms). For the first iteration of measuring the Malaysian AI Ethics Maturity, the researchers are focused on two key assessments of transparency and explainability; and, privacy and data governance.
[1] Artificial Intelligence in the Asia-Pacific Region: Examining policies and strategies to maximise AI readiness and adoption, International Institute of Communications, Artificial Intelligence in the Asia-Pacific Region, February 2020, p 8.
[2] https://www.microsoft.com/en-us/ai/responsible-ai
[3] https://ai.google/principles/
[4] https://rm.coe.int/ethical-charter-en-for-publication-4-december-2018/16808f699c
[5] https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai
[6] https://www.microsoft.com/en-us/ai/responsible-ai
2. THE SURVEY
The Survey was carried out in collaboration with BIGIT. The Survey document was prepared by the researchers and administered by BIGIT as a questionnaire.
The researchers wish to emphasise that the Survey is a dipstick survey that was undertaken to draw early insights into assessing the degree of Trustworthy AI against the standard of BDA Maturity * and AI Maturity* amongst Deployers of AI tools which are viewed as ethical. The outcome of the Survey is an AI Ethical Maturity index ranked by industries.
The Survey questions have been adopted and adapted from the Trustworthy AI Assessment List prepared by the European Commission’s High-Level Expert Group on Artificial Intelligence (AI HLEG)’s document titled Ethics Guidelines for Trustworthy AI published in April 2019. The Survey will focus on one of these 5 key assessments – namely, transparency and explainability, and privacy and data governance.
The Survey comprised Survey 1 and Survey 2. Survey 1 consists of 22 questions requiring either a “Yes” or “No” answer that is divided into two parts - Part A and Part B.[7] Part A required responses to questions as to why the Respondents have decided to use AI in their organisation’s decision-making processes and Part B comprised two key assessments relating to, firstly, transparency and explainability; and, secondly, privacy and data governance respectively of the said AI systems. The Survey aimed to enquire into the extent of good practices and measures adopted by Deployers of AI in ensuring an ecosystem of trust in the AI system lifecycle.
The survey questions have been adopted and adapted from the “Trustworthy AI Assessment List” prepared by the European Commission’s High-Level Expert Group on Artificial Intelligence (AI HLEG)’s document titled “Ethics Guidelines for Trustworthy AI” published in April 2019. The survey will focus on one of these 5 key assessments – namely, privacy and data governance.
Survey 2 consists of five questions requiring answers to be provided on a scale of 1 to 5 which are aimed at gauging the Respondents’ opinion on the importance of strengthening the ecosystem of Trustworthy AI. The purpose of this Survey is to aid the researchers to measure the organisational attitudes of the Deployers of AI, and to gauge their perception of the importance of strengthening an ecosystem of Trustworthy and Responsible AI in terms of implementation of AI Governance in policy at the national level and within organisations.
For the first iteration of measuring the Malaysian AI Ethics Maturity, as stated above, the researchers focussed on two key assessments with the Respondents comprising Deployers of AI systems.
[7] Part B Section 2 of the Survey 1 was used as part of the research activity of a grant titled ‘Recommendations for the creation of a governance framework for the protection of personal data used in the development of AI systems’ awarded to the researchers by the Malaysian Communications and Multimedia Commission (MCMC). The researchers wish to acknowledge the support provided by MCMC.
3. METHODOLOGY
3.1 Research Design and Measurement
3.1.1 Survey 1
The research design is a non-experimental correlational quantitative Survey. This Survey was done through an ad hoc instrument consisting of a total of 22 structured response format items (questions) which were placed in two parts. Part A of the Survey comprises three questions (Questions 1 to 3) to assess the degree of Deployers’ organisation’s awareness of the use of the system. Part B of the Survey has two components. Part B Section 1 of the Survey comprises two Pillars, of which Questions 4 to 8 measure the degree of explainability and transparency adopted by Deployer (Pillar 1) and Questions 9 to 14 measure the extent of communication to individuals* as part of assessing explainability mechanisms (Pillar 2). Part B Section 2 of the Survey comprises three Pillars, of which Questions 15 to 17 measure Deployers’ respect for privacy and data protection (Pillar 1), Questions 18 to 20 measure their control of quality and integrity of data (Pillar 2), and Questions 21 and 22 measure their approach on access to data (Pillar 3).
The researchers aimed to measure the variables within each question, component, and Pillar to assess the statistical relationship by calculating the frequency of the “Yes” and “No” responses to determine the relationship between the two variables and the direction of the relationship between the two variables. The items in the instrument are adapted from the pilot version of the assessment list for Trustworthy AI provided by the European Commission in its Ethics Guidelines for Trustworthy Artificial Intelligence document that was developed in close collaboration with stakeholders across the public and private sectors by the European Commission’s High-Level Expert Group on AI (the AI HLEG). All items in the instrument were answered on a dichotomous Yes/No scale. A Glossary of Terms can be found at the end of this report.
3.1.2 Survey 2
The research design is a quantitative survey. This Survey was done through an instrument consisting of five close-ended items in total, of which items 1 to 3 measure perception on the role of policy and policymakers; item 4 measures perceptions on the importance of training and education; and item 5 measures perception on soft governance.* This instrument was answered on a 5-point Likert scale from 1(not important) to 5 (very important) with 3 (neutral) as the mid-point. The data will be analysed using descriptive statistics.
3.2 Participants and Data Collection
This questionnaire was administered to a purposive sampling of 200 AI Deployers (the intended sample size was approximately 100 Deployers of AI in organisations across Malaysia that collect and use personal data in AI systems who will be selected through a purposive sampling method and participation from amongst this sample was on a voluntary and self-selection basis) determined by the administrator of the Survey. The Survey was administered as a questionnaire determined by our collaborator, BIGIT. Data collected was provided to the researchers. The total number of Respondents is 103 which were then clustered into industries.
The Respondents were categorised according to industries as stated below. Respondent industries with less than three Respondents (Agriculture & Mining; Consumer Goods, Education; and Oil & Gas) were removed as they are not sufficiently representative for analysis. In addition, the Aviation industry was merged with Transportation & Logistics. The final industry count is twelve and these industries set out as follows with description of each industry (provided by the research collaborator, BIGIT):
Computers & Electronics: The manufacturing of electronics, creation of software, computers, or products and services relating to information technology.
Energy & Utilities: Generation, transmission, distribution, and retailing of electricity; distribution and retailing of natural gas; water services provision.
Financial Services: Provision of financial services to people and corporations. Made up of a variety of financial firms including banks, investment houses, lenders, finance companies, real estate brokers, and insurance companies.
Government: All units of central, state or local government; all social security funds at each level of government; all non-market non-profit institutions that are controlled and financed by government units.
Healthcare: Companies that offer clinical services, manufacture drugs and medical equipment, and provide healthcare-related support services like medical insurance.
Manufacturing: Production of goods through the use of labour, machines, tools, and chemical or biological processing or formulation.
Media & Entertainment: Collection of businesses that allows information to be shared. This includes operations such as radio broadcasts, websites, and newspapers.
Professional Services: Professional services include lawyers, advertising professionals, architects, accountants, financial advisers, engineers, and consultants, among others. They are an organisation or profession that offers customized, knowledge-based services.
Real Estate & Construction: Development, leasing, appraisal, marketing, and management of commercial, residential, agricultural, and industrial properties
Software & Internet: Development, maintenance, and publication of software products used for different business models, either licensed or cloud-based.
Telecommunications: Cable companies, internet service providers, satellite companies, and telephone companies offering communication services over a distance.
Transportation & Logistics: The science of obtaining, producing, and distributing material and products to the correct place and in the correct quantities.
3.3 Data Analysis
The data was analysed using descriptive statistics, in particular the measure of central tendency and the measure of Standard Deviation. The researchers analysed the data by measuring the Mean through the distribution of “Yes” responses for each item within each industry. The percentage of “Yes” responses within each industry was plotted against all the twelve industries to measure Standard Deviation.
4. ANALYSIS OF SURVEY I
4.1 Part A: Utility of System
This part of the Survey comprises three questions. These questions aim to assess the Deployer’s degree of awareness in utilising an AI system in particular why the system was used, its value to the organisation, and whether there was information relayed to individuals of the purpose and benefit of the AI system.
a. Ranking of Industries
Table A.1 represents the ranking of the industries through a sequential colour palette in accordance with the percentage of “Yes” responses to the questions within the specific industry for the question in this part of the Survey. The data visualisation through a colour palette facilitates the ease in identifying where each industry is placed in terms of the highest percentage and the lowest percentage of “Yes” responses. Graph A.2 represents the ranking of industries in accordance with the total percentage of “Yes” responses from all three questions. Graph A.3 indicates the number of Yes/No responses of the industries to the questions in this part of the Survey.
b. Survey Questions and Responses
Question 1: Did you know why this particular system was deployed in this specific area?
Graph A.4 indicates the number of Yes/No responses of each industry to Question 1. The question seeks to determine the relationship between the utility of the AI system by the Deployer and the reason for the decision in the utility of the AI system. The relationship between the two variables, indicative through the “Yes” responses, demonstrates the rationalisation for the utility of the AI system as part of the decision-making process before its use.
Question 2: Did you know the business model concerning this system (e.g. how it creates value for the organisation)?
Graph A.5 indicates the number of Yes/No responses of each industry to Question 2. The question seeks to determine the relationship between the utility of the AI system by the Deployer and the business model concerning the system. The relationship between the two variables, indicative through the “Yes” responses, demonstrates once again the rationalisation for the utility of the AI system as part of the decision-making process before its use as its use must not be for the case of “tech for tech-sake” but must create value for the delivery of services by the organisation to individuals.
Question 3: Did you make clear to users what the purpose of the AI system is and who or what may benefit from the product/service?
Graph A.6 indicates the number of Yes/No responses of each industry to Question 3. The question seeks to determine the relationship between the utility of the AI system by the Deployer and the duty to inform the users (the customers, clients, or parties with whom the Deployer is engaging with and these parties will be impacted by the use of the AI system) of the use and benefits of the system. The relationship between the two variables, indicative through the “Yes” responses, demonstrates the importance of organisations that are taking advantage of the use of AI systems in their processes to notify stakeholders of the purpose and benefits of the system.
c. Average and Standard Deviation
Graph A.7 below was plotted based on a Mean of 74% with a standard deviation of 17.6.
4.2 Part B: Section 1: Transparency and Explainability
This part of the Survey comprises two Pillars – Explainability and Communication. Transparency of the data, system, and AI business models is important to ensure good governance. Ensuring that the development and deployment process embraces mechanisms that ensure traceability* and explainability mechanisms can foster greater confidence in the use of AI-related products and services. By creating awareness that individuals are interacting with AI and enabling those affected by an AI system to understand the outcome, the capabilities of AI systems, and their limitations will ensure a process of feedback that may be adapted to improve these systems and avoid potential challenges to its use and latent risks.
4.2.1 Ranking of Industries
Table B.1.1 represents the ranking of the industries through a sequential colour palette in accordance with the percentage of “Yes” responses to the questions within the specific industry for each individual Pillar. The data visualisation through a colour palette facilitates the ease in identifying where each industry is placed in terms of highest percentage and the lowest percentage of “Yes” responses. Graph B.1.2 represents the ranking of industries in accordance with the total percentage of “Yes” responses across both Pillars. Graph B.1.3 indicates the number of Yes/No responses of the industries to the questions in Part B Section 1 of the Survey.
4.2.2 Pillar 1 - Explainability
a. Survey Questions and Responses
Question 4: Did you know the extent to which the outcome made by the AI system can be understood?
Graph B.1.5 indicates the number of Yes/No responses of each industry to Question 4. The question seeks to determine the relationship between explainability to ensuring the understandability of the decisions produced by the AI system. The relationship between the two variables, indicative through the “Yes” responses, demonstrates the importance of internal mechanisms in place as part of governance measures in ensuring that that the requirement of explainability is satisfied.
Question 5: Did you ensure that an explanation as to a certain outcome can be made understandable to all users that may desire an explanation?
Graph B.1.6 indicates the number of Yes/No responses of each industry to Question 5. The question seeks to determine the relationship between explainability and the ability to ensure that the outcome is explainable and understandable to stakeholders. The relationship between the two variables, indicative through the “Yes” responses, demonstrates the requirement of explainability extends to the understandability of the outcome to external parties who may be impacted by the outcome.
Question 6: Did you design the AI system with interpretability in mind from the start?
Graph B.1.7 indicates the number of Yes/No responses of each industry to Question 6. The question seeks to determine the relationship between explainability and the integration of the requirement of interpretability of how the AI system predicts. The relationship between the two variables, indicative through the “Yes” responses, evidences the importance of developing an AI by design to ensure that it upholds the necessary values.
Question 7: Did you assess whether you had the opportunity to examine the interpretability of the trained and developed AI model?
Graph B.1.8 indicates the number of Yes/No responses of each industry to Question 7. The question seeks to determine the relationship between explainability and the interpretability of the AI system’s prediction. The relationship between the two variables, indicative through the “Yes” responses, evidences the understanding of the performance of the AI system to improve and develop a robust algorithm.
Question 8: Did you have access to the internal workflow of the model?
Graph B.1.9 indicates the number of Yes/No responses of each industry to Question 8. The question seeks to determine the relationship between the requirement of explainability and review of the internal workflow of the algorithm’s functionality. The relationship between the two variables, indicative through the “Yes” responses, evidences the importance of the need to improve the AI system during its lifecycle through continuous development of the algorithm.
b. Average and Standard Deviation
Graph B.1.10 below was plotted based on a Mean of 70% with a standard deviation of 12.8.
4.2.3 Pillar 2 - Communication
a. Survey Questions and Responses
Graph B.1.11 indicates the overall Yes/No responses of the industries to all 6 questions in Pillar 2.
Question 9: Did you communicate to individuals – through a disclaimer or any other means – that they are interacting with an AI system and not with another human?
Graph B.1.12 indicates the number of Yes/No responses of each industry to Question 9. The question seeks to determine the relationship between transparency and communication of the use of AI systems to individuals in respect of their engagement with an AI system. The relationship between the two variables, indicative through the “Yes” responses, evidences the importance of the need to communicate to individuals that an AI system is being used as opposed to exclusively human interaction.
Question 10: Did you label your AI system as such?
Graph B.1.13 indicates the number of Yes/No responses of each industry to Question 10. The question seeks to determine the relationship between transparency and communication of the use of AI systems to individuals by labelling clearly that an AI system is being used. The relationship between the two variables, indicative through the “Yes” responses, evidences the importance of individuals being clear that an AI system is being used at the outset.
Question 11: Did you put in place mechanisms to inform individuals of the reasons and criteria behind the AI system’s outcomes? Is this clearly and intelligibly communicated to the individuals?
Graph B.1.14 indicates the number of Yes/No responses of each industry to Question 11. The question seeks to determine the relationship between transparency and communication of the reason and criteria utilised in the algorithm to the outcome in a clear and intelligible manner to individuals. The relationship between the two variables, indicative through the “Yes” responses, evidences the degree of transparency in the use of the AI system.
Question 12: Did you establish processes that take into account individuals’ feedback, relaying it to the Developer for use of this feedback to adapt and/or improve the system?
Graph B.1.15 indicates the number of Yes/No responses of each industry to Question 12. The question seeks to determine the relationship between transparency and communication from individuals in the form of feedback as to their interaction with the AI to continue improving the AI during its lifecycle. The relationship between the two variables, indicative through the “Yes” responses, demonstrates the continuous improvement of the AI through reciprocal mechanisms as part of efforts to improve the algorithm’s functionality.
Question 13: Did you also communicate around potential or perceived risks, such as bias?
Graph B.1.16 indicates the number of Yes/No responses of each industry to Question 13. The question seeks to determine the relationship between transparency and communication of the potential risk of the use of AI systems in predicting outcomes. The relationship between the two variables, indicative through the “Yes” responses, evidences the importance of accountability on the part of the Deployer of the potential risks that are inherent in the system.
Question 14: Did you clearly communicate characteristics, limitations and potential shortcomings of the AI system to the individuals?
Graph B.1.17 indicates the number of Yes/No responses of each industry to Question 14. The question seeks to determine the relationship between transparency and communication of the characteristics of the AI system used by the Deployer and its limitations in terms of its use in the use case* as well as its shortcomings. The relationship between the two variables, indicative through the “Yes” responses, the openness and transparency on the part of the Deployer of relaying the extent of effectiveness of the AI system.
b. Average and Standard Deviation:
Graph B.1.18 below was plotted based on a Mean of 64% with a standard deviation of 13.9.
4.3 Part B: Section 2: Privacy and Data Governance
4.3.1 Ranking of Industries
Table B.2.1 represents the ranking of the industries through a sequential colour palette in accordance with the percentage of “Yes” responses to the questions within the specific industry for each individual Pillar. The data visualisation through a colour palette facilitates the ease in identifying where each industry is placed in terms of the highest percentage and the lowest percentage of “Yes” responses.
For Pillar 1, Table B.2.1 indicates the percentage of the industries based on the percentage of “Yes” responses where there is the highest degree of respect for privacy and data protection which is indicated through the establishment of mechanisms (i) to flag up issues relating to data protection and privacy, (ii) for notice and control over personal data and (iii) the involvement of a data privacy officer in the deployment of the AI system.
For Pillar 2, Table B.2.1 indicates the percentage of the industries where there is the highest degree of measures to ensure quality and integrity of data based on the percentage of “Yes” responses.
For Pillar 3, Table B.2.1 indicates the percentage of the industries where there is the highest degree of measures in place to control access to personal data in line with the Security principle of the PDPA based on the percentage of “Yes” responses.
4.3.2 Pillar 1 - Respect for Personal & Data Protection
a. Ranking
Since data is vital in the life cycle of AI systems, the interactions of AI systems with data, both individual and collective, must be undertaken in a manner that upholds privacy and data protection principles. This includes ensuring the law of the land is complied with, as well as ensuring good data governance practices.
b. Survey Questions and Responses
Question 15: Depending on the use case, did you establish mechanisms that allow others to flag issues related to privacy or data protection issues concerning the AI system’s processes of data collection (for training as well as operation) and data processing?
Graph B.2.4 indicates the number of Yes/No responses of each industry to Question 1. The question seeks to determine the relationship between the establishment of mechanisms of flagging privacy or data protection issues and the respect for personal and data protection. The relationship between the two variables indicative through the “Yes” responses establishes the fact that certain industries are more aware of the need to respond to concerns raised by customers.
Question 16: Did you build in mechanisms for notice and control over personal data depending on the use case (such as valid consent and the possibility to revoke, when applicable)?
Graph B.2.5 indicates the number of Yes/No responses of each industry to Question 16. The question seeks to determine the relationship between the implementation of notice and choice principle through mechanisms in line with the statutory data protection principle of notice and choice and respect for personal data protection. The relationship between the two variables is indicative through the “Yes” responses of the extent to this principle is respected.
Question 17: Was an officer responsible for data privacy involved in the deployment of the AI system?
Graph B.2.6 indicates the number of Yes/No responses of each industry to Question 17. The question seeks to determine the relationship between the assigned role in the organisation responsible for data privacy for the deployment of an AI system to the extent there is respect within the organisation for privacy and data protection. The relationship between the two variables indicative through the “Yes” responses reveals the importance of this office for addressing not only concerns raised by customers but also the importance of resolving them.
c. Average and Standard Deviation:
Graph B.2.7 was plotted based on a Mean of 63% with a standard deviation of 23.
4.3.3 Pillar 2 - Quality & Integrity of Data
a. Ranking of Industries
To ensure the quality and integrity of the data, adoption of a framework with established oversight mechanisms is fundamental and management of the quality of the data when resourced from external data sources.
b. Survey Questions and Responses
Question 18: Is the system aligned with the principles of the Personal Data Protection Act (Malaysia) and widely adopted protocols for data privacy i.e. GDPR and ISO 27701/27001?
Graph B.2.9 indicates the number of Yes/No responses of each industry to Question 18. The question seeks to assess the relationship of ensuring the quality and integrity of data with the alignment of fundamental data protection principles. The relationship between the two variables, indicative through the “Yes” responses, provides a measure of the extent to the national data protection principles and/or other instruments play a role in the quality and integrity of data which includes accuracy and security of the data.
Question 19: Did you establish oversight mechanisms for data collection, storage, processing and use?
Graph B.2.10 indicates the number of Yes/No responses of each industry to Question 19. The question seeks to assess the relationship between the establishment of oversight mechanisms of the data lifecycle in ensuring the quality and integrity of data. The relationship between the two variables, indicative through the “Yes” responses, is essential in ensuring that oversight mechanisms function as a layer of responsibility and accountability employed to ensure that is proper governance.
Question 20: If you are using external data in the AI system are you in control of the quality of the external data sources used?
Graph B.2.11 indicates the number of Yes/No responses of each industry to Question 20. The question seeks to determine the relationship between the quality and integrity of the data used in the training of the AI system through the exercise of a form of control on the quality of the external data. The relationship between the two variables, indicative through the “Yes” responses, shows that there is a degree of oversight in ensuring that large external data sets that are used are accurate as they may impact the quality of the predictions made by the AI system.
c. Mean and Standard Deviation
Graph B.2.12 below was plotted based on a Mean of 68% with a standard deviation of 17.
4.3.4 Pillar 3 - Access to Data
a. Ranking of Industries
In protecting the personal data of data subjects, there needs to be measures in place to ensure that access to data must be controlled in terms of who has permission to access that data within the organisation and the competencies of this person. Graph B.2.13 indicates the overall Yes/No responses of the industries to all three questions in Pillar 3.
b. Survey Questions and Responses
Question 21: Did you assess who can access individuals’ data, and under what circumstances?
Graph B.2.14 indicates the number of Yes/No responses of each industry to Question 21. The question seeks the relationship between the levels of access to data held by the organisation to the accessibility of the data by personnel and the conditions the person can access the said data. The relationship between the two variables that is indicative through the “Yes” responses evidences that there are restrictions on the accessibility of the data to ensure security and to avoid unnecessary disclosure in line with data protection principles.
Question 22: Did you ensure that these persons are qualified and required to access the data, and that they have the necessary competencies to understand the details of data protection policy?
Graph B.2.15 indicates the number of Yes/No responses of each industry to Question 22. The question seeks to determine the relationship between the accessibility of the data and the standard of competency of the individuals accessing the data. The relationship between the two variables indicative through the “Yes” responses is that there are restrictions and thresholds in place which distil who has the competency to ensure that any data protection privacy in place is adhered to.
c. Mean and Standard Deviation
Graph B.2.16 below was plotted based on a Mean of 75% with a standard deviation of 18.
5. SURVEY 1: FINDINGS AND ANALYSIS
For each part of the Survey, a general section on Mean and Standard Deviation is set out followed by, depending on the different components of the Survey, pillar-specific observations. The findings and analysis of Survey 1 culminate in an index of AI Ethics Maturity in Graph C.2 for each Industry with reference to the BDA Maturity and AI Maturity ranking of the industries according to the Malaysia AI Blueprint Annual Report for 2021.
5.1 Part A: Utility of System
a. Mean and Standard Deviation – General Remarks
Within the industries, the overall number of “Yes” responses are overwhelmingly on the upper side with a Mean of 74%. This evidence a high degree of awareness amongst Deployers. Anomalies appear concerning responses across all three questions for two industries - Manufacturing and Transportation & Logistics where “Yes” responses for Questions 1 and 2 were lower in comparison to that of Question 3. This raises concerns of the congruence on the rationalisation of the use of AI systems within the organisation, and the extent of communication of its use to individuals. With 100% “Yes” responses in Healthcare, Professional Services, and Real Estate & Construction, this is indicative of a high level of confidence in the use and deployment of AI in the said industries. The only industry that has a higher number of “No” responses in comparison to “Yes” responses is Transportation and Logistics.
Across all industries, except for Media & Entertainment, there is a higher number of “No” responses to Question 1 in comparison to the other two questions in this part of the Survey. It is indicative of a weakness in the rationalisation of the utility of the AI system.
In industries with a high ranking in the BDA Maturity and AI Maturity indices, such as Transportation & Logistics and Telecommunication, there is a need for strengthening this rationalisation to communicate and inform individuals of the purpose and benefit of the use of AI systems in delivering services.
5.2 Part B: Section 1: Transparency and Explainability
a. Mean and Standard Deviation – General Remarks
The researchers observed the Mean and Standard Deviation for each pillar in relation to the industries. Amongst the two Pillars, Pillar 2 on Communication has the lower Mean of 64% (with a Standard Deviation of 13.9) in comparison to Pillar 1 on Explainability which has a Mean of 70% (with a Standard Deviation of 12.8). Several industries, namely, Telecommunications, Media & Entertainment, and Transportation & Logistics scored consistently low on the “Yes” responses across both Pillars raising obvious questions around employment of good governance from an ethical persepctive. Manufacturing demonstrates the highest deviation between the three Pillars. Professional Services, Real Estate & Construction, and Healthcare had the highest number of “Yes” responses indicative of governance measures and mechanisms in promoting the value assessed in this part of the Survey. Manufacturing demonstrates a large difference between the percentage of “Yes” responses between the two Pillars – ranked 3rd highest for Pillar 1 and ranked 2nd from the bottom for Pillar 2. The high deviation between the two Pillars indicates that there are industries that may have processes in place in ensuring explainability but may not be communicating the use of the AI systems to individuals. The shortfall in the parallelism between the two Pillars similarly appears in other industries’ findings. The industries scoring a percentage of “Yes” responses that exceeds the Mean for Pillar 1 are Government, Healthcare, Professional Services, Real Estate & Construction, Manufacturing, and Software & Internet. This finding lends credence to the fact that there is a significant number of industries - in this case half of the 12 industries - that demonstrate a degree of awareness on the importance of explainability and interpretability. For Pillar 2, the industries scoring the percentage of “Yes” responses exceeding the Mean are Real Estate & Construction, Professional Services, Healthcare, and Government. The outcome for this Pillar is disappointing as it indicated that communication of the use of the AI system which lends to the value of transparency of the said use amongst individuals who are stakeholders such as consumers was inadequate. From the findings, the most apparent concern was that Financial Services fell below the Mean for both Pillars. Industries at the top of the BDA and AI Maturity ranking such as Telecommunications, Media & Entertainment, Financial Services, and Transportation and Logistics ranked low across both Pillars – the only exception being Healthcare – echoing the same concern.
b. Pillar-specific observations across all industries
Pillar 1- Explainability
Questions 4 and 5 received a high percentage of “Yes” responses indicating that the understandability of the AI system is viewed as important by the industries signifying organisational understanding of the AI system including understandability of the outcome of the use of the AI system to external parties. For Question 4, Real Estate & Construction and Manufacturing had a remarkable 100% “Yes” responses that reveals that there are mechanisms in place as part of governance measures adopted by organisations within the industry to have sufficient insight into the workings of the algorithm. Question 5 has a marginally lower percentage of “Yes” responses. Whilst several industries such as Telecommunications have a low “Yes”response percentage for this question, Energy & Utility, Financial Services, and Software & Internet has an increase in the same. This evinces the importance of the understandability on the part of individuals as to the outcome of the AI and how the AI reached the same. There is a lower percentage of “Yes” responses for Questions 6 and 7 which assessed the aspect of interpretability. Interpretability, as part of the ethical principle of requiring AI systems to be explainable to assess the veracity of the outcome of an AI system, is compromised if organisations are not advancing its importance in the AI lifecycle. On the question of accessibility to the internal workflow of the AI system in Question 8, Healthcare, Government and Real Estate & Construction evidences a 100% “Yes” response. However, for Telecommunication, the lower percentage of “Yes” responses remains prevalent as it is the only industry that indicates a higher number of “No” responses in comparison to “Yes” responses. Access to internal workflow will allow Deployers to assess the veracity of the workings and output of the AI system during its lifecycle to detect any errors, risks, or biases. For Pillar 1 of this component of the Survey in relation to transparency and explainability, Financial Services fell below the Mean for this Pillar and equally concerning was Telecommunication’s average of “Yes” responses places it second from the bottom (see Table B.1.1). The findings where industries have fallen below the Mean to the questions within this Pillar will relegate the industry in terms of its ranking in the AI Ethics Maturity index.
Pillar 2- Communication
For Questions 9 and 10, there is a higher percentage of “No” responses in two industries - Software & Internet and Transportation & Logistics. For Question 10, on labelling the AI system, Financial Services has an increased number of “No” responses whereas responses of communicating to individuals* that AI systems are being used has a higher average of “Yes” responses. The premise of these questions was to determine the extent communication of the use of AI systems to individuals was undertaken and the findings lead the researchers to conclude that in these industries, the said practise needs to be implemented, and if already implemented, for it to be further strengthened. For Question 11, in the assessment of processes in place in informing individuals of the reason and criteria for the use of AI system in clear terms, Transportation & Logistics fares poorly amongst the industries with more “No” responses. Further, for the same question, Financial Services has a marginally higher “Yes” response at 54% than other questions but still narrowly lower than the Mean. The outcome of this question reveals where industries are falling behind on the transparency element in order to curtail the issue of opacity of AI systems. On the question of processes of receiving feedback to improve the AI system, which is Question 12, Financial Services fares better evidencing mechanisms in place for this purpose. Question 13 and 14 has a positively higher percentage of “Yes” responses in relation to communicating potential or perceived risk as well as the characteristics, limitations and potential shortcomings of the AI system to individuals. The findings present a good trend in upholding transparency but this needs further exploration as to how the industries are communicating risks and potential shortcomings of the AI System to individuals. This is necessary to assess its adequacies as it appears to be inconsistent with responses in Question 11 which assesses the mechanisms used to communicate the reasons and criteria behind the AI system.
5.3 Part B: Section 2: Privacy and Data Protection
a. Mean and Standard Deviation – General Remarks
The researchers observed the Mean and Standard Deviation for each pillar in relation to the industries. The lowest Mean is for Pillar 1 (with a higher deviation of 23) and the highest Mean is for Pillar 3 (with a lower deviation of 18). The industry that is consistently low on the “Yes” responses across all three Pillars with the lowest Mean is Transportation & Logistics. Weak data governance practices or perhaps weak confidence and trust in the sufficiency of such practices could explain the low response. Large deviations between Pillars in the percentage of “Yes” responses could be seen across Pillars - where Manufacturing demonstrates the highest deviation between the three Pillars. Media & Entertainment demonstrates a large difference between the Mean in Pillar 1 and the Mean in Pillars 2 and 3. This reveals that measures have been taken in ensuring respect for personal data but not comprehensively or consistently across the Pillars. The top five industries including the Government scoring the highest percentage of Yes responses are Professional Services, Real Estate & Construction, Healthcare, and Financial Services. All of these five industries exceed the Mean for all three Pillars. This is indicative of good data governance practices. The remaining industries generally fell below the Mean in all three Pillars, except for Energy & Utilities, Software & Internet and Manufacturing which respectively reach or exceed the Mean in only one Pillar.
b. Data Intensive Sectors
A large number of industries fall within the Class of Data Users listed in the Personal Data Protection (Class of Data Users) Order 2013. These include communications, banking, and financial institutions, insurance, health, tourism and hospitality, transportation, education, direct selling, real estate, and utilities. In fact, four of these sectors have registered codes of practices – utilities(electricity), communications, banking and financial sector, and insurance and takaful.
To interpret and explain the findings of the research, the researchers embarked on investigating whether the industries fell within the class of data users under the Personal Data Protection (Class of Data Users) Order 2013 and the Personal Data Protection (Class of Data Users) Amendment Order 2016. The researchers would define “Data Intensive Sectors” (DIS) based on this class as the determination of this class requires the data users to register with the Data Protection Commissioner. This is indicative of the named data users carrying out activities that envisage a significant volume of personal data collection that the PDPA aims to regulate or that the risk attached to the personal data collected may be significant. Since the PDPA does not extend to government bodies, the said Orders do not name government authorities within the class. For the purposes of analysis of the Survey, the researchers are of the view that Government can easily be classified as a DIS owing to the extent of data collection undertaken in carrying out the administrative function of ministries, departments, agencies and local councils. Table C.3 indicates the extent the industries involved in the Survey fall within the class of data users. This indication is important to the researchers as it may explain the extent of data protection practices that are aligned with the overarching data protection principles in the PDPA and the maturity of these practices within the respective industry. Table C.3 indicates whether an individual industry falls within the class of data users listed in both the above orders in the following manner:
· Where the industry falls within a stipulated class, the table indicates a “Yes” affirmation with the √ symbol.
· Where the industry does not fall within a stipulated class, the table indicates a “No” affirmation with the X symbol.
· Where the industry falls in part within a stipulated class, the table indicates a “Partial” affirmation with the ∂ symbol.
c. Review of Industry Responses which are DIS
Amongst the top five industries with the highest “Yes” responses, three of these are DIS, namely Professional Services, Healthcare, and Financial Services. Financial Services indicates a high level of “Yes” responses likely because these series of questions are closely linked to compliance with the PDPA required within the industry. It is important to make mention that Financial Services has its own registered Code of Practice. Question 20 in Pillar 2 relating to whether the Respondents had control of the quality of the external data sources appeared problematic for the top four industries (Professional Services, Financial Services, Real Estate & Construction, and Healthcare) which fares weakly with a higher percentage of “No” responses. Energy & Utilities fares slightly lower than the other five DIS industries when it did not exceed the Mean. This is particularly the case in Pillar 2. Responses of Media & Entertainment and Transportation & Logistics across the Pillars are consistently low in the “Yes” responses. The Telecommunication industry which falls within the “Communication” class of data users fares a low percentage of “Yes” responses amongst all three Pillars, falling below the Mean for each Pillar even where it has adopted a Code of Practice. For example, a surprising finding is the industry’s response to Question 15 indicating a lack of compliance with the PDPA in terms of the AI system. If the industry is in the class and scores a low Mean, this indicates a worrying trend.
d. Review of Industry Responses which are non-DIS
Real Estate & Construction is not a DIS but is amongst the top five industries with the highest percentage of “Yes” responses (see Graph B.2.2). This is demonstrative of good standards of self-governance. Computer & Electronics, Software & Internet and Manufacturing fall below the Mean for all three Pillars. The researchers are able to rationalise this position as they are not DIS. For example, Manufacturing shows lower “Yes” responses (50% margin) in comparison to other parts of the questionnaire. For instance, to the question, if they had designated an officer responsible for privacy, the “Yes” responses are a meagre 25%. The fact that it may not be mandatory for the industry to have a privacy officer as Manufacturing does not fall within the Class of Data Users could explain the industry’s responses to the Survey. Software & Internet and Manufacturing fared well in Pillar 3 equalling the Mean and exceeding it, respectively. Computer & Electronics’ answers to Question 18 (refer to Graph B.2.9) has a high ratio of “Yes” to “No” answers (4:1). This is not representative of the said industry’s answers to other questions in the Survey i.e. ratio of 3:2 “Yes” to “No” responses. It indicates an awareness by the said industry of a need/desire for compliance which is not translating to adequate implementation.
e. Pillar-specific observations across all industries
Pillar 1- Respect for Privacy and Data Protection
There is a higher number of “Yes” responses for this pillar amongst DIS industries in comparison with non-DIS industries. This is evidenced in Graph B.2.4 and Graph B.2.8. This clearly indicates that where there is an obligation of compliance with the PDPA on a particular industry when it is classified as a DIS, the degree of respecting data protection principles is observed. The DIS status of an industry as a DIS seems to have an impact on the positive responses corresponding to the level of respect for privacy and data protection.
Pillar 2 - Quality & Integrity of Data
Amongst the “Yes” responses in this Pillar, the responses to Question 20 demonstrate a lower rate of “Yes” responses. This rate is evidenced with a lower rate of “Yes” responses in comparison to Questions 18 and 19 (Financial Services and Manufacturing), or, a higher number of “No” responses in comparison to “Yes” responses (Energy & Utilities, Media & Entertainment, Software & Internet). The industries with this lower rate of “Yes” responses to Question 20 fall within the categories that are DIS, non-DIS, and partially DIS. The classification of DIS may not be relevant as the industries are not utilising personal data from data subjects with whom they have a relationship as a data user since the use of data in the AI system is externally sourced. The responses to this question are extremely important as it is indicative of the control the industries have on the quality of external data sources that are being used to train the AI system that these industries are deploying. With most of the ethical issues around AI system centering around biases in the large data sets used to train the algorithm, managing the quality of the external data sources points towards resolving a fundamental dilemma in ensuring that AI is trustworthy.
Pillar 3 - Access to Data
There is a high number of “Yes” responses for both questions within Pillar 3 across all industries. This is evidenced by the Mean of 75 being the highest of the three Pillars as seen in Graph B.2.16. The overall positive responses are seen across the industries in Graph B.12.15 with only the Transportation industry having a higher percentage of “No” response to Question 22. By looking at the prevailing responses, it is indicative of a high level of confidence across the industries of compliance with the Security principle of the PDPA in the context of internal access to personal data in AI systems employed.
5.4. AI Ethics Maturity
The researchers reviewed the overall “Yes” responses in Survey 1 and according to the percentage of the “Yes” responses, has ranked the industries in an AI Ethics Maturity index as seen in Graph C.2. The criteria for the AI Ethics Maturity index are the extent of awareness of the utility of the AI system; the degree of transparency and explainability; and the degree of respect for privacy and adoption of data governance measures. Graph C.3 compares the AI Ethics Maturity ranking with that of two other indices – the AI Maturity ranking and the BDA Maturity ranking as found in the Malaysia AI Blueprint Annual Report for 2021. The AI Ethics Maturity index which is the ranking of the industries in the adoption of ethical measures does not evidence an alignment with the Deployers’ BDA Maturity and AI Maturity rankings. The only industry that holds the same position across all three indices is Software & Internet which is ranked 7th. Financial Services, Healthcare, Computers & Electronics, and Manufacturing indicate a minimal variation in the ranking across all three indices. Media & Entertainment, Transportation & Logistics, and Telecommunications which are ranked high on the BDA Maturity and AI Maturity indices, evidenced extremely low ranking on the AI Ethics Maturity index. Industries lower on the BDA Maturity and AI Maturity ranking such as Professional Services, Real Estate & Construction, Government and Energy & Utilities ranked higher in the AI Ethics Maturity index which indicates a more responsive behaviour towards ethical principles. The opposed positions of these industries across the indices appear to reveal an interesting phenomenon. Regardless, the researchers are not impervious to the findings’ revelation that there is a need for an improved alignment between the good governance of adopting measures that operationalise ethical principles.
6. SURVEY 2: FINDINGS AND ANALYSIS
Survey 2 titled “Policy Guidance in Strengthening an Ecosystem of Trustworthy AI” required the participation of Respondents in order for the researchers to gauge the perception of the importance of strengthening an ecosystem of Trustworthy AI in terms of implementation of AI Governance through the formulation of policies at the national level. Table D sets out the five (5) questions posed to the Respondents in Survey 2. Respondents were required to answer on a Likert scale of 1 to 5 (the category of ‘1’ being “Not Important” and ‘5’ being “Very Important”). Graph E below sets out the total number of answers for each question on the scale of 1 to 5.
Total answers for “Very Important” are highest across all five questions. For Questions 1 and 2, the dominant answer is “Important” whereas, for Questions 3, 4, and 5, the dominant answer is “Very Important”. To be also noted is a significant percentage of “Neutral” answers to Question 5 which translates to 26%. The findings highlighted above indicate a strong sentiment across the industries towards strengthening an ecosystem of Trustworthy AI in terms of implementation of AI Governance policies at the national level. In particular, Respondents indicate a desire to see the formation of a National Level Expert Group that harmonises the development of AI with value-based principles (Question 3), and the provision of training and guidance by the government on trustworthy AI to operationalise the principles being pursued (Question 4). The responses to Question 5 however indicate a hesitance from several Respondents towards implementation of a certification process.
7. LIMITATIONS
Below are the limitations of this Survey:
The Survey was intended to be a ‘Dipstick’ study to assess the adoption of ethical measures to draw some early insights into assessing the level of maturity of this adoption and compliance amongst Deployers of AI tools.
The Survey was limited to Deployers and did not include Developers of AI systems.
The Survey was limited to two key assessments namely Transparency & Explainability, and Privacy and Data Governance.
The Survey was issued and responses were collected by the research collaborator. As such, the procedure adopted in sending out the Survey and retrieving the responses was not entirely in the control of the researchers.
The definitions and categorisation of the industries in the Survey were determined by the research collaborator.
The primary data does not allow the researchers to investigate the selection of responses by the Respondents of the Survey. Further studies require a more detailed questionnaire for future investigation that is experimental which may include semi-structured interviews with participants.
8. Future Research
The following are considerations for future research and the next iteration of this survey.
8.1 Respondents
The researchers aim to consider an enlarged category of Respondents to include Developers.
8.2 Survey 1
The researchers aspire to enlarge the current key assessments to include the remainder of the five key assessments based on value-based principles that are robustness, security and safety; human-centred values, fairness and societal well-being; and accountability. The adoption of these additional criteria will enhance the findings of the AI Ethics Maturity assessment and ranking of the industries.
a. Part A: Utility of System
The researchers aspire to undertake a more detailed probing on the rationale for the use of AI by Deployers and the measures employed by Deployers to create awareness amongst individuals in particular the impact the use of AI systems may have on individuals.
b. Part B: Section 1: Transparency and Explainability
To review the emerging judicial propositions and advisement resulting from the adjudication in the courts on the importance of overcoming the opacity of AI through transparency and explainability. The researchers’ review of the trends in the courts in cases being litigated will require a recalibration of questions in this section of the Survey.
The researchers note that organisations around the world are exploring new technologies and methods that attempt to provide increased explainability in AI systems to meet the rising demand for such explainability and transparency. Future research may consider the extent to which Malaysia employs such technologies or methods i.e. post-hoc explanation models, local interpretability models for black box systems and Layer Wise Relevance Propagation (LRP).
c. Part B: Section 2: Privacy and Data Governance
At the time of developing the Survey question, the researchers relied on the European Commission’s High-Level Expert Group on Artificial Intelligence (AI HLEG)’s document titled “Ethics Guidelines for Trustworthy AI”. There have been recent legislative initiatives concerning the employment of high-risk data practices in the development and deployment of AI tools.
These include the Data Protection Act of 2021 introduced as a US Bill in the Senate and the European Commission Proposal referred to as the Artificial Intelligence Act.
These initiatives indicate that regulatory frameworks are moving towards governance structures that require an assessment of risk to determine a classification scheme of AI tools. This classification scheme will categorise AI tools to the extent of risk which is as follows prohibited, high-risk, low-risk, and no-risk.
8.2 Survey 2
In relation to the high number of “neutral” answers to Question 5 of Survey 2 that is demonstrating a hesitancy of the certification process, the researchers are of the view that this aspect requires further exploration as this hesitancy may be embedded in several views including possibly a perception that such certification would be too rigorous and hence, act as a hindrance to the development, deployment of AI tools.
9. Concluding Remarks
The Survey has identified that:
Need to build a higher degree of Deployers’ awareness in rationalising the use of AI systems in their organisations particularly in industries where use is pervasive;
Need to have improved measures to increase explainability which includes understandability and interpretability as well as ensuring transparency of how the AI systems are employed and its impact on individuals in particular in industries that are indicating a high permeation of use of AI tools;
Need to explain how the industries are communicating risks and potential shortcomings of the AI System to individuals to assess its adequacies;
Need to explore and explain several anomalies within the DIS and AI Maturity measurements. By anomalies, the researchers have found that there were industries that were categorised as DIS that did not perform well in the adoption of ethical principles, and conversely, in non-DIS, there were indications of good ethical practices;
Need for an improved alignment by adopting measures that promote ethical governance as there was no congruence between the adoption of ethical principles with the BDA and AI Maturity indices of the industries as evidenced in the AI Ethics Maturity index; and
Need to strengthen the ecosystem of Trustworthy AI through the adoption of policies both at the national and organisational level as part of AI Governance initiatives. There is evidence that Deployers are in favour of a move in this direction.