Towards NLP. Страница 4
1054
@towards_nlp
NLP: все n-граммы про анализ текстов. По всем дополнительным вопросам:
-
Towards NLP
Stable Diffusion ...is very powerful and impressive and it is worth it to dedicate some time to get know what is it. If you just want to play and generate images: DreamStudio If you want to learn what the model is about: tutorial If you want to download weight of the model: 🤗 model card If you want to dive in and play with code: Colab Notebook The whole press-release with all useful links: Stability AI page the fancy restaurant at the edge of the planet with universe sky 
-
Towards NLP
-
Towards NLP
The most important news due today: I got access to DALLE-E by OpenAI. And it is really super interesting to play with text prompts! But, yes, you need to be quite precise with a text request. Prompt tuning (even human) can be our everything. My requests: * main character is a nlp programmer surrounded by logos bert, huggingface, pytorch, openai, word2vec, fasttext and space background; * the inside of the mind of dall-e by openai; * a cool researcher sitting in the modern office in front of the laptop surrounded by numbers and formulas and a cat in a style of pop-art; Rate the accordance😃 To get access as well: join waitlist here. 
-
Реклама
-
Towards NLP
PIXEL Tokenization is one of the main steps in the preprocessing pipeline of the text before it comes as input to a model. Language models are defined over a finite set of inputs, which creates a vocabulary bottleneck, it is hard to propagate the approach of tokeniaztion when you want to work with multiple languages. How to overcome this problem? We can refer to text as to an image and work not with tokens but with patches (as in image processing). The idea is based on ViT-MAE — a Transformer-based encoder-decoder trained to reconstruct the pixels in masked image patches. So, the text can be rendered as an image and then go to the Encoder layer avoiding Embedding layer. Thus, theoretically, such model can support any language (even if in the texts there are some emojis or any other pictured symbols, as on the screenshot) as you can mask anything! The paper: https://arxiv.org/pdf/2207.06991.pdf The code: https://github.com/xplip/pixel The already pretrained models on 🤗: https://huggingface.co/Team-PIXEL 
-
Towards NLP
Distilled NLLB is now in 🤗 Distilled version of Meta AI model for 200+ languages translation is now available here. -
Towards NLP
BigScience Large Open-science Open-access Multilingual Language Model Or just BLOOM🌸 New multilingual autoregressive Large Language Model. Main info: * Base model: Megatron-LM GPT2; * Architecture: decoder-only; * N parameters: 176B (in comparison: for example, mBART-cc25 has 610M parameters); * N languages: 59; * How can be used?: text generation or exploration of languages characteristics and dependencies; All other details and the model itself can be found in 🤗 here. 🌼🌸🌺Enjoy🌼🌸🌺 -
Towards NLP
No Language Left Behind: Scaling Human-Centered Machine Translation Quite important news for multilingual models and machine translation — Meta AI aiming to make machine translation research better released its models to opensource🎉 "200+ languages — including low-resource languages like Asturian, Luganda, Urdu and more" What models are released: * SOTA model of 54.5B parameters; * Dense versions pf 3.3B and 1.3B parameters; * And distilled versions of 1.3B and 600M parameters respectively. Code of models, metrics are also released here! Sounds amazing✨ -
Towards NLP
My ACL 2022: Diversity and Equality..? ACL 2022 was hold in the end of May and I had a possibility to participate there offline. After the years of pandemic the experience of offline conference was quite cool, however the in the end impressions for me were controversial. One of the biggest "meh" was the whole line of "diversity" in English-speaking conference where 90% of works are dedicated to English language. Personally I, a scientist from Eastern Europe, born in Ukraine, felt quite as from an underrepresentative group in all this English-speaking society. While, the work that I was presenting was about English language, what an irony! The picture above tries to illustrate the "diversity" of affilations and countires published in ACL. Do you agree that it is quite diverse for the conference dealing with language? As a result, all these statements about such diverse ACL community sounds like a joke to me. 
-
Towards NLP
But, in the end, I liked the conference in the way that this is a good opportunity to look at a lot of NLP-scientific directions and find your own way of identification as a researcher. That is why, I would like to also share a personal update - from June, 2022 I am obtaining a research position at TUM in Social Computing Research Group working with topic "Explainable Hate Speech Detection". Moreover, as you can notice, this channel did not always cover all "hype" NLP-topics. In my future scientific career path I would like to focus on NLP for Social Good topics as well as multilingual NLP. So, the posts in this channel will represent my scientific interests. If you are interested to try to make NLP technologies indeed good, please, join me in this path! 
-
Towards NLP
CMU Multilingual NLP Course For everyone who is interested in processing of multilingual text and speech data, now there is available the course from Carnegie Mellon University of 2022 year. Enjoy! youtube.com/playlis…playlist -
Towards NLP
ParaDetox: Detoxification with Parallel Data At this ACL I was presenting our current culmination of the detoxification project — parallel dataset with pairs "toxic sentence <-> non-toxic paraphrase" together with the pipeline of such dataset collection. What for? Now detoxification task can be solved as a typical machine translation task that allows to achieve quite good quality of text style transfer models. Moreover, pipeline of dataset collection can be used for any other text style transfer task. What we release: * ParaDetox dataset in HuggingFace🤗 repo; * New SOTA model for detoxification also in 🤗 here; All other details can be found in our github repo. You are very welcomed to play with our detoxifier! 
-
Towards NLP
April NewsLetter This newsletter covers PaLM, DALL-E 2, and Chinchilla, chain-of-thought prompting, and the role of values and culture in NLP. by Sebastian Ruder here -
Towards NLP
Big step after first DALL·E — DALL·E 2 In January 2021, OpenAI introduced DALL·E. One year later, their newest system, DALL·E 2, generates more realistic and accurate images with 4x greater resolution. The first DALL·E is a transformer model. It receives both the text and the image as a single stream of data containing up to 1280 tokens, and is trained using maximum likelihood to generate all of the tokens, one after another. This training procedure allows DALL·E to not only generate an image from scratch, but also to regenerate any rectangular region of an existing image that extends to the bottom-right corner, in a way that is consistent with the text prompt. In the second DALL·E they reformated method and now it is CLIP + diffusion model. CLIP to encode text prior and diffusion model to decode resulting embeding to high resolution image. So it’s simply GLIDE, but with some tweaks. To generate high resolution images, they train two diffusion upsampler models. But the results are amazing. Despite that it is cherry picks of course :)) - paper - blog with images and demos - video 
-
Towards NLP
Fake News As I personally have been developing the technology for fake news detection and the current situation has caused the explosion of fake news production, I want to share my research and the technology which I am personally using for fake news check. (it is scientific NLP channel anyway) My paper: Cross-lingual Evidence Improves Monolingual Fake News Detection [link] The idea. if you read/find some piece of news of its title: 1. cross-check it in incognito mode in any search you using — so, "google" it, find at least 5 trustable news sources that are saying absolutely the same information. Also, it would be great if textual information is supported with some visual one. 2. Do the same for several other languages. Now, for example, Google Translate works extremely well and it is possible to accurately translate news titles and articles. There is even a new function to translate the whole websites. So, translate the original news into some language (for example, language of your neighbor country), "google" again, copy paste founded title into Google Translate and check the facts. I am inserting the schema of the proposed approach, hope, it will clarify the algorithm. Disinformation is also a weapon nowadays. Fight it as well. -
Towards NLP
Models based on graphs are quite important for a lot of tasks in NLP. There is an overview from Michael Bronstein about what he is expecting for upcoming year for the Graph ML field: 1. Geometry becomes increasingly important in ML. 2. Message passing is still the dominant paradigm in GNNs. 3. Differential equations give rise to new GNN architectures. 4. Old ideas from Signal Processing, Neuroscience, and Physics get a new life. 5. Modeling complex systems requires going beyond graphs. 6. Reasoning, axiomatisation, and generalisation are still big open questions in Graph ML. 7. Graphs become increasingly popular in Reinforcement Learning, but probably still have a way to go. 8. AlphaFold 2 is a triumph of Geometric ML and a paradigm shift in structural biology. 9. Drug discovery and design benefits from GNNs and their confluence with Transformers. 10. AI-first drug discovery is increasingly using Geometric and Graph ML. 11. Quantum ML benefits from graph-based methods. [link] 
-
Реклама
-
Towards NLP
-
Towards NLP
Controllable Neural Text Generation How to generate text in desirable style? How to train such model? Is RL applicable to this task? Here is a long-read about how you can improve your text generation strategies and adjust for your task. 1. Common Decoding Methods. 2. Smart Prompt Design. 3. Fine-Tuning. 4. RL Fine-Tuning. Here: lilianweng.github.io/lil-log…ion.html -
Towards NLP
RUSSE-2022 Russian Detoxification Test Phase As one of the organizers of Russian Detoxification tasks, I would like to invite you to participate in our last test phase! This is the unique competition of Text Style Transfer task with parallel corpora and automated manual evaluation via crowd-sourcing. Do not miss you chance to compete for the best detoxification system💪 All the data and code for baselines could be found in our git repo: https://github.com/skoltech-nlp/russe_detox_2022. Specifically, here is the test set. You need to make a submission in the test phase in Codalab and submit your solution with description via google-form before 13, February 23:59. We will evaluate all solutions. The only condition — meaningful description of your solution. Any other question you can ask in our Telegram group and other news in our Telegram-channel. Good luck⭐️