Transformers for Natural Language Processing_ Build, train and fine-tune deep neural network architectures, 2nd Edition_Denis Rothman.pdf

Transformers for Natural Language Processing Second Edition Build, train, and fine-tune deep neural network architectures for NLP with Python, PyTorch, TensorFlow, BERT, and GPT-3 Denis Rothman BIRMINGHAM—MUMBAI Transformers for Natural Language Processing Second Edition Copyright © 2022 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Producer: Tushar Gupta Acquisition Editor – Peer Reviews: Saby Dsilva Project Editor: Janice Gonsalves Content Development Editor: Bhavesh Amin Copy Editor: Safis Editing Technical Editor: Karan Sonawane Proofreader: Safis Editing Indexer: Pratik Shirodkar Presentation Designer: Pranit Padwal First published: January 2021 Second edition: March 2022 Production reference: 3170322 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-80324-733-5 www.packt.com Foreword In less than four years, Transformers took the NLP community by storm, breaking any record achieved in the previous 30 years. Models such as BERT, T5, and GPT, now constitute the fun- damental building bricks for new applications in everything from compute