Singapore: Singapore Launches World's First AI Governance Testing Framework and Toolkit

In brief

On 25 May 2022, Singapore launched the World's first AI Governance Testing Framework and Toolkit ("AI Verify"), which is intended for companies wishing to demonstrate responsible AI in an objective and verifiable manner. This is currently available as a Minimum Viable Product (MVP) for system developers and owners who want to be more transparent about the performance of their AI through a combination of technical tests and process checks.

The launch of AI Verify forms part of Singapore's greater commitment to remain a leading and active participant in the furtherance of AI Ethics.


Contents

It complements past publications such as: (1) the Model AI Governance Framework ("Model Framework"), which provides detailed and ready-to-implement guidance to private sector organizations on how to address key ethical and governance issues when deploying AI solutions; (2) its accompanying Implementation and Self-Assessment Guide to help organizations assess the alignment of their internal processes with the Model Framework; and (3) a Compendium of Use cases to showcase the successful implementation of Model Framework measures.

Organizations would do well to participate in piloting the MVP in order to gain early access to its self-testing capabilities, use MVP-generated reports to demonstrate transparency, and help develop an MVP that accurately reflects industry needs and contributes to the development of international standards.

The Pilot Testing Framework and Toolkit does not, however, guarantee that any Al system tested under this Pilot Framework will be free from risks or biases or is completely safe.

In more detail

On 25 May 2022, Singapore launched AI Verify, the World's first AI Governance Testing Framework and Toolkit, which is intended for companies wishing to demonstrate responsible AI in an objective and verifiable manner. This is currently available as an MVP for system developers and owners who want to be more transparent about the performance of their AI through a combination of technical tests and process checks.

Broadly, developers and owners can verify the claimed performance of their AI systems against a set of principles through standardized tests. Self-assessment is carried out using a toolkit that comprises a package of open-source testing solutions and is capable of generating reports for developers, management, and business partners, covering major areas affecting AI performance. The areas covered are aligned with desiderata set out in internationally accepted AI ethics, principles, guidelines and frameworks. These include, but are not limited to:

  1. Transparency:
    1. On the use of AI to achieve what stated outcome
    2. Understanding how the AI model reaches a decision
    3. Whether the decisions predicted by the AI show unintended bias
  2. Safety and resilience of AI system.
  3. Accountability and oversight of AI systems.

As it is in the nascent stages of development, the MVP currently:

  • Works with a certain subset of common AI models but does not currently support unsupervised models;
  • Can handle tabular datasets but has limited support for image datasets; and
  • Supports small-to-medium scale models (around two GB) that can be fully imported using a web interface.

Related articles

Singapore: The High Court issues injunction to block potential sale and transfer of NFT

Singapore: PDPC publishes guide on responsible use of biometric data in security applications

Singapore: Government proposes Codes of Practice to regulate harmful online content on social media

 

LOGO_Wong&Leow_Singapore

© 2022 Baker & McKenzie.Wong & Leow. All rights reserved. Baker & McKenzie.Wong & Leow is incorporated with limited liability and is a member firm of Baker & McKenzie International, a global law firm with member law firms around the world. In accordance with the common terminology used in professional service organizations, reference to a "principal" means a person who is a partner, or equivalent, in such a law firm. Similarly, reference to an "office" means an office of any such law firm. This may qualify as "Attorney Advertising" requiring notice in some jurisdictions. Prior results do not guarantee a similar outcome.


Copyright © 2023 Baker & McKenzie. All rights reserved. Ownership: This documentation and content (Content) is a proprietary resource owned exclusively by Baker McKenzie (meaning Baker & McKenzie International and its member firms). The Content is protected under international copyright conventions. Use of this Content does not of itself create a contractual relationship, nor any attorney/client relationship, between Baker McKenzie and any person. Non-reliance and exclusion: All Content is for informational purposes only and may not reflect the most current legal and regulatory developments. All summaries of the laws, regulations and practice are subject to change. The Content is not offered as legal or professional advice for any specific matter. It is not intended to be a substitute for reference to (and compliance with) the detailed provisions of applicable laws, rules, regulations or forms. Legal advice should always be sought before taking any action or refraining from taking any action based on any Content. Baker McKenzie and the editors and the contributing authors do not guarantee the accuracy of the Content and expressly disclaim any and all liability to any person in respect of the consequences of anything done or permitted to be done or omitted to be done wholly or partly in reliance upon the whole or any part of the Content. The Content may contain links to external websites and external websites may link to the Content. Baker McKenzie is not responsible for the content or operation of any such external sites and disclaims all liability, howsoever occurring, in respect of the content or operation of any such external websites. Attorney Advertising: This Content may qualify as “Attorney Advertising” requiring notice in some jurisdictions. To the extent that this Content may qualify as Attorney Advertising, PRIOR RESULTS DO NOT GUARANTEE A SIMILAR OUTCOME. Reproduction: Reproduction of reasonable portions of the Content is permitted provided that (i) such reproductions are made available free of charge and for non-commercial purposes, (ii) such reproductions are properly attributed to Baker McKenzie, (iii) the portion of the Content being reproduced is not altered or made available in a manner that modifies the Content or presents the Content being reproduced in a false light and (iv) notice is made to the disclaimers included on the Content. The permission to re-copy does not allow for incorporation of any substantial portion of the Content in any work or publication, whether in hard copy, electronic or any other form or for commercial purposes.