Deepfake porn case raises alarm over digital safety, says Teo

A recent case involving a secondary-school student who used deepfake technology to create pornographic images of his schoolmates and school alumni in Johor, has highlighted the urgent need for stricter digital safety protocols in all education institutions, deputy communications minister Teo Nie Ching reportedly said. 

The minister added that one of the most pressing issues that requires attention, is the attitude and response of schools toward such complaints, noting that schools must act with seriousness and urgency, reported The Star.

Teo also said that this is a message that must be sent to all social media users, and to institutions entrusted with protecting students. She added that in this case, the victim had informed the school prior to filing a police report, but the complaint was not treated with appropriate seriousness. 

Don't miss: MCMC investigates political influencer and one other over 3R content

Separately, Teo said in a statement on Facebook that 40 victims of the deepfake porn case have been identified, with the youngest being just 13 years old. Of these, 16 victims lodged police reports on Saturday (12 April) under the Kulai constituency office's accompaniment. Teo said that most of the victims were still in school and only learned about the existence of the doctored images after being informed by the school. 

"Some victims revealed they had discovered these images as early as 2023, but did not step forward to make a report at that time. We hope that more victims who have yet to lodge a report will find the courage to come forward and file a police report," said Teo.

The deputy communications minister who is also an MP of Kulai, held a press conference with six victims who bravely stepped forward to face the media. 

"Their courage deserves our full support and recognition. They are refusing to let the reprehensible actions of others dictate their future or sense of self-worth. However, it is important to remember that not all victims may feel as confident or empowered as they do. In fact, it is precisely because of their bravery that more young women have started to share their experiences on social media," added Teo. 

"This reminds us that incidents of technology-facilitated gender-based violence are not isolated. What we are seeing is likely just the tip of the iceberg. Schools must play their part by establishing clear guidelines to manage such complaints more effectively in the future," she said. 

Teo further explained that in November 2023, the education ministry released a guideline for managing sexual misconduct in educational institutions, which clearly state that the school must contact a victim's parents or guardians and lodge police report within 24 hours of receiving a complaint. "While these guidelines do not cover private, international, or independent schools, they remain highly valuable and relevant as a reference," she said.

"Ultimately, we must stand united in our schools, in our homes, and in our communities to create a culture where our children are protected, survivors are supported, and perpetrators are held accountable. Every young person deserves to grow up in an environment where they feel safe, respected, and heard," added Teo. 

The issue came to light last week, when a 16-year-old boy was arrested for allegedly using AI to create lewd images of several girls from his school and selling them. The images were created using the victims' social media images. According to a report by China Press, each image was priced at RM2 each on social media platforms. 

The boy, who has been in police custody amidst investigations, has also been expelled from his school. According to The Vibes, the suspect's remand has been extended until tomorrow (15 April), as his motives are still under investigation. 

Earlier in February, communications minister Fahmi Fadzil said that some 1,993 pornographic social media posts and advertisements were reportedly removed in Malaysia from 2022 until February this year by the Malaysian communication and multimedia commission (MCMC). Fahmi also said that in the same period, 3,670 websites were blocked for inappropriate content.

He added that once the Online Safety Act is in force, the government will emphasize the need for social media platform providers to prevent prohibited advertisements and materials from being broadcast. 

The Online Safety Bill was passed in Malaysian parliament on 16 December 2024 and is expected to come into force at a later date, complementing the Communications and Multimedia Act. The bill applies to social media services and messaging apps, as well as content applications like video streaming apps, and network services.

At the end of last year, the MCMC was reported to have removed 1,225 items of AI-generated explicit content as of 1 December 2024, compared to 186 in 2022. Teo said that the MCMC has removed 800 items of child sexual exploitation content, 766 more than in 2022, according to a Bernama report. 

Join us this coming 21 May for #Content360, an extravaganza centered around three core thematic pillars: Challenging the norm; Conquering Media Fragmentation; and Unlocking Imagination. Immerse yourself in learning to curate content with creativity, critical thinking, and confidence with us at Content360!   

Related articles:
Nearly 2,000 porn posts and ads removed by MCMC since 2022, says Fahmi
MCMC removed 8,756 cyberbullying-related content in 2024, five times more than 2023
MCMC removes over 1,000 AI-generated explicit content, up from 186 in 2022