On Feb. 24, VNG ran a big headline "municipalities reconsider deployment of Microsoft Copilot." Upon closer inspection, the article turned out to be only about the municipality of Amsterdam. They have decided not to start a proposed pilot with Copilot. The earlier version of the article also had to be corrected because it stated that the municipality had stopped a pilot. So it was only an intention. What exactly the pilot was, is unfortunately not clear from the reaction of the municipality of Amsterdam in the article. In any case, this report brought renewed attention to the privacy risks that had already been identified in a DPIA for Copilot last December.
On behalf of SLM Rijk, the Privacy Company conducted a DPIA on Microsoft 365 Copilot. The public version thereof was published on December 17 last year. To be clear, the DPIA and thus this blog is only about the privacy risks, not the risks of sharing business-sensitive or - secret information.
As is well known, Copilot is a generative AI service from Microsoft. Among other things, Copilot can help create summaries, texts, conversations and calculations in Microsoft's major applications: Word, Excel, PowerPoint, Outlook and Teams. Microsoft Enterprise users have been automatically logged into Copilot since mid-September last year. The DPIA was performed on the paid Enterprise license.
The main difference between the free versions of Copilot and the Enterprise version with Data Protection is that the paid Enterprise license has access to so-called Graph information. Graph allows Copilot to retrieve relevant information from your work context, such as:
Recent documents you have worked with
Emails or chats related to a project
Agenda appointments and meeting notes.
Organizational structure and colleagues you work with
Copilot uses Microsoft Graph to:
Provide contextual answers: For example, if you ask, "What were the action items from my last meeting?", Copilot can retrieve notes from your Teams meeting.
Find files and documents: For example, if you ask, "Show me the latest version of the marketing plan," Copilot can show the appropriate OneDrive or SharePoint files.
Summarize emails and conversations: Copilot can summarize recent emails or Teams chats on a specific topic.
Both the free and paid versions of Copilot generate answers based on information from the OpenAI Large Language Model (LLM) and Bing-based web chat.
The outcome of this DPIA is that there are 4 high and 6 low privacy risks when using Copilot. I only dwell on the high risks here , you can read about the 6 low risks in the DPIA itself.
Three of the four high risks relate to a lack of transparency about the collection of required service data (called Required Service Data). These are the minimum data required for a service or service to function correctly. According to Microsoft, at Copilot, these include:
User Information: Identification information such as your account ID and licensing information.
Device and software data: Operating system, application version and settings.
Interaction data: How you use Copilot, such as what features you turn on.
Diagnostic data: Errors, crashes and performance data.
Service communication: Logs about how Copilot communicates with other Microsoft services.
The fourth high risk is the risk common to all LLMs: the risk of incorrect and incomplete personal data in the generated responses.
The DPIA identifies the following privacy risks.
Data subject's rights, including the right to inspect required service data is inadequate. Inspection requests did not yield clear answers.
Loss of control over personal data due to lack of transparency on required service data.
The risk of re-identification of pseudonymized data due to unknown retention periods of required service data.
Economic or social disadvantages due to the use of generated texts with incorrect personal data.
The agreement that the Dutch government has with Microsoft stipulates that Microsoft has the role of processor for the personal data it processes. This DPIA therefore shows that Microsoft independently processes data outside the instructions of the controller and is not transparent about it. Thereby, the processing is not compatible with the purposes for which government organizations allow Microsoft to collect personal data.
SLM Empire and the IBD (InformatieBeveiligingsDienst van de VNG) therefore advise against using Copilot until Microsoft has mitigated the four high risks. Last December Surf (the ict cooperative of education and research) also already advised its members not to use Copilot for the time being.
SLM Empire, by the way, has been in in discussions with Microsoft about mitigating the 4 high risks.
Both parties provide an opinion, it is of course up to organizations themselves to weigh whether or not to use Copilot based on the risks from the DPIA.
