Defining Django 2.0 urls.py robot.txt File

To define the robots.txt file in Django 2.0, you can add the following code to the urls.py file.

path('robots.txt', lambda x: HttpResponse("User-Agent: *\nDisallow:", content_type="text/plain")),