Обложка канала

Towards NLP

NLP: все n-граммы про анализ текстов. По всем дополнительным вопросам:

Towards NLP

5 лет назад
Открыть в
Inappropriateness detection In addition to the aforementioned model detecting toxicity in Russian texts, my colleagues collected the dataset of inappropriate utterances and trained the model on it. The 'inappropriateness' is not a substitution of toxicity, it is rather a derivative of toxicity. These are namely the utterances on sensitive topics (the topic which potentially yields unsafe discussion, such as racism, sexism, drugs, LGBT etc) which can, for example, harm the reputation of the company if its dialogue agent replies accordingly. In the same time it is unwanted to block any mention of such sensitive topic by brute-force keywords approach. The published model makes the first step to perform such smart filtering in Russian language. Come and try it!) If you want to learn more about the collected dataset you can learn more in this light-reading article on Skoltech website or in the article published on ACL workshop. P.S. So, there samples in the comments that sometimes current toxicity detection model labels 'toxic' vulnerable topics. We believe, in the future both approaches will be combined in the robust toxicity classifier.
Skoltech/russian-inappropriate-messages · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co