EPFL, ETH Zurich and the Swiss National Supercomputing Centre (CSCS) released Apertus today, Switzerland’s first large-scale, open, multilingual language model — a milestone in generative AI for transparency and diversity.

First “modern and powerful” open source LLM?

Key features

  • Fully open model: open weights + open data + full training details including all data and training recipes
  • Massively Multilingual: 1811 natively supported languages
  • Compliant Apertus is trained while respecting opt-out consent of data owners (even retrospectivey), and avoiding memorization of training data
  • lime!@feddit.nuEnglish
    13·
    3 days ago

    that’s the problem with deletion requests, the data isn’t in there. it can’t be, from a purely mathematical standpoint. statistically, with the amount of stuff that goes into training, any full work included in an llm is represented by less than one bit. but the model just… remakes sensitive information from scratch. ih reconstructs infringing data based on patterns.

    which of course highlights the big issue with data anonymization: it can’t really be done.