Testing is a crucial aspect of software development, particularly in web development. Even a single broken link or endpoint can result in a poor user experience or security vulnerabilities. Luckily, Django offers powerful testing tools, including the Django test framework, which can be enhanced with third-party packages like pytest to tackle these challenges.
When dealing with large, long-standing systems, it's common to have numerous endpoints. Many of these endpoints may be infrequently used but still necessary. Unfortunately, they can easily slip from our minds when implementing changes, leading to unforeseen impacts on users that go undetected.
In this developer blog post, we'll dive into an approach for testing all the endpoints of a Django server. We'll utilize the "show_urls" command from django-extensions and the "get" method of pytest's django_app_factory plugin. Together, these tools will enable us to systematically test each endpoint. Join us as we walk through the entire process, providing a detailed explanation of how it all works.
Getting All Endpoints
The show_urls command is a useful tool provided by django-extensions that offers a list of all endpoints in JSON format. Each entry returned has four fields: url, module, name, and decorators. By iterating over the returned list, we can test each endpoint with a GET request to identify broken links and ensure that all endpoints are functioning properly.
To implement this test, we can start by using Django's call_command method to run the show_urls command and get a list of all endpoints.
Identifying Maintained Endpoints
The first step after acquiring all endpoints is to sort them based on whether they are maintained or not. This can be achieved by checking the module field that contains “path” for the endpoint, because the first part of the path is the app name.
Having the app name, we can now easily check if the app is maintained or not, and skip the ones that are not.
Building the URL and Necessary Data
Next, let's focus on constructing the URL for our GET request. To ensure we have all the necessary data, we'll need to consider adding query parameters and keyword arguments to the URL field. In Django, we can achieve this using the url field and leverage the Django syntax.
For keyword arguments, there are many that are generated by django as a model. These models follow a pattern that we can easily use and replace for its intended value. But there are some that don’t follow this pattern which are mostly custom endpoints that need to be checked individually to add their value.
Unfortunately, there is no easy way to check query parameters as there is no denotation for them on the url, so they need to be checked individually by looking into the code, usually after a request fails.
In this step we need to create a high amount of pytest fixtures too as most of the values needed are model identifiers or are related to them.
Performing the GET Request
Once we have built the URL and necessary data, we can perform the GET request using pytest's django_app_factory plugin's get() method. In this step, more fixtures may be required and there's also a possibility of internal errors occurring. I had some occurrences of implementation errors from old endpoints in production showing some good results from these tests already.
There is not much that can be done here other than waiting for a response and following to the next step or handling the errors that happen from time to time.
Checking the Response
Finally, we need to check the status code returned by the GET request. If the code is outside the 2xx range and is not 405, indicating that the endpoint does not allow GET requests, the test will fail unless this endpoint is explicitly listed as expecting another code. The request follows redirects, so we should not receive any 3xx code either.
Conclusion
In conclusion, thoroughly testing all the endpoints of a Django server is a crucial step in guaranteeing the smooth functioning of a web application and delivering a positive user experience. Fortunately, with the powerful Django test framework and the aid of valuable third-party packages such as django-extensions and pytest, developers can effortlessly create comprehensive tests. By leveraging these tools, broken links and endpoints can be identified, and potential errors can be caught before they have a chance to impact users.
