Django源码分析--路由分发

Posted on | In

上一节，我们分析了Django的网络服务的启动流程，以及它如何通过网络层接收到数并将请求转发给Django Application。那么这一节，我们主要分析，当Request到达Django Application之后，Django内部的路由规则是如何将其导入到正确的View（视图）方法中的呢？

1、准备工作

  • Python 3.5.2
  • Django 2.1.2
  • PyCharm 2018.2.1 (Professional Edition)
  • 启动项目
1
[min:] ~/Desktop/python/Demo$ python manage.py runserver 0.0.0.0:8000

2、分析流程

django.core.handlers.base.BaseHandler#get_response

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
def get_response(self, request):
    """Return an HttpResponse object for the given HttpRequest."""
    # Setup default url resolver for this thread
    set_urlconf(settings.ROOT_URLCONF)

    response = self._middleware_chain(request)  # 重点关注！！

    response._closable_objects.append(request)

    # If the exception handler returns a TemplateResponse that has not
    # been rendered, force it to be rendered.
    if not getattr(response, 'is_rendered', True) and callable(getattr(response, 'render', None)):
        response = response.render()

    if response.status_code >= 400:
        log_response(
            '%s: %s', response.reason_phrase, request.path,
            response=response,
            request=request,
        )

    return response
  • 从Django源码分析–服务启动中，我们知道了当请求到达时，最后会调用到WSGIHandler的__call__方法，然后会执行response = self.get_response(request)，即调用父类中的get_reponse方法；
  • 在此方法中我们重点关注一下response = self._middleware_chain(request)，发现该属性来自于load_middleware方法。

django.core.handlers.base.BaseHandler#load_middleware

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
def load_middleware(self):
    """
    Populate middleware lists from settings.MIDDLEWARE.

    Must be called after the environment is fixed (see __call__ in subclasses).
    """
    self._view_middleware = []
    self._template_response_middleware = []
    self._exception_middleware = []

    handler = convert_exception_to_response(self._get_response)
    for middleware_path in reversed(settings.MIDDLEWARE):
        middleware = import_string(middleware_path) # 通过类似反射的方法获得对象实例；
        try:
            mw_instance = middleware(handler)   # 调用MiddlewareMixin的__init__方法；
        except MiddlewareNotUsed as exc:
            if settings.DEBUG:
                if str(exc):
                    logger.debug('MiddlewareNotUsed(%r): %s', middleware_path, exc)
                else:
                    logger.debug('MiddlewareNotUsed: %r', middleware_path)
            continue
				########省 略######
        handler = convert_exception_to_response(mw_instance)

    # We only assign to this when initialization is complete as it is used
    # as a flag for initialization being complete.
    self._middleware_chain = handler
1
2
3
4
5
6
7
8
9
10
11
12
13
14
class MiddlewareMixin:
    def __init__(self, get_response=None):
        self.get_response = get_response
        super().__init__()

    def __call__(self, request):
        response = None
        if hasattr(self, 'process_request'):
            response = self.process_request(request)
        # 这步比较关键，决定了会按照MIDDLEWARE中的顺序依次倒序执行；
        response = response or self.get_response(request) 
        if hasattr(self, 'process_response'):
            response = self.process_response(request, response)
        return response

  • 这个方法就很重要，如果大家有看过我们的上一个章节，就会知道这个方法是WSGIHandler的__init__方法中调用的，也就是在执行runserver命令的时候就执行的；

  • convert_exception_to_response是一个装饰器，通过这装饰器和MiddlewareMixin方法结合，使得最后的

    handler实例可以依次倒序执行 settings.MIDDLEWARE中配置的所有的中间件方法，最后执行_get_response方法！

    备注：这块比较考察对装饰器的理解，需要好好琢磨一下，关键点在于理解在于MiddlewareMixin的get_response对应的就是类似于SecurityMiddleware的实例，所以每次__call__方法中response = response or self.get_response(request)就相当于重新调用了新的Middleware的__call__；

django.core.handlers.base.BaseHandler#_get_response

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
def _get_response(self, request):
    """
    Resolve and call the view, then apply view, exception, and
    template_response middleware. This method is everything that happens
    inside the request/response middleware.
    """
    response = None

    # 获得Django App的Root URLResolver
    if hasattr(request, 'urlconf'):
        urlconf = request.urlconf
        set_urlconf(urlconf)
        resolver = get_resolver(urlconf)
    else:
        resolver = get_resolver()
	
  	# 根据URL完成匹配，返回一个ResolverMatch示例
    resolver_match = resolver.resolve(request.path_info)
    # 分解ResolverMatch示例，得到对应的view方法，即callback
    callback, callback_args, callback_kwargs = resolver_match
    request.resolver_match = resolver_match

    # Apply view middleware
    for middleware_method in self._view_middleware:
        response = middleware_method(request, callback, callback_args, callback_kwargs)
        if response:
            break

    if response is None:
        wrapped_callback = self.make_view_atomic(callback)
        try:
            # 重要，在此处完成了对view方法对执行；
            response = wrapped_callback(request, *callback_args, **callback_kwargs)
        except Exception as e:
            response = self.process_exception_by_middleware(e, request)

    # Complain if the view returned None (a common error).
    if response is None:
        if isinstance(callback, types.FunctionType):  # FBV
            view_name = callback.__name__
        else:  # CBV
            view_name = callback.__class__.__name__ + '.__call__'

        raise ValueError(
            "The view %s.%s didn't return an HttpResponse object. It "
            "returned None instead." % (callback.__module__, view_name)
        )

    # If the response supports deferred rendering, apply template
    # response middleware and then render the response
    elif hasattr(response, 'render') and callable(response.render):
        for middleware_method in self._template_response_middleware:
            response = middleware_method(request, response)
            # Complain if the template response middleware returned None (a common error).
            if response is None:
                raise ValueError(
                    "%s.process_template_response didn't return an "
                    "HttpResponse object. It returned None instead."
                    % (middleware_method.__self__.__class__.__name__)
                )

        try:
            response = response.render()
        except Exception as e:
            response = self.process_exception_by_middleware(e, request)

    return response
  • 这个方法是整个路由分发系统的核心，可以看到就是在这个方法，request对应的视图方法被匹配到，执行并返回结果，即URLResolver根据path info匹配到正确的ResolverMatch，然后调用其中的view方法，并返回respone，具体的过程看上述代码注释，接下来我们进入详细分析；
  • 还有一个需要说明的就是在这个方法中判断respone中有没有render，有就调用；
    1
    2
    3
    4
    5
    @functools.lru_cache(maxsize=None)
    def get_resolver(urlconf=None):
        if urlconf is None
            urlconf = settings.ROOT_URLCONF
        return URLResolver(RegexPattern(r'^/'), urlconf)

tutorial/settings.py

1
ROOT_URLCONF = 'tutorial.urls'

tutorial/urls.py

1
2
3
4
5
6
urlpatterns = [
    url(r'^', include('snippets.urls')),
    url(r'^api-auth/', include('rest_framework.urls', namespace='rest_framework')),
    url(r'^schema/$', schema_view),
    url(r'^docs/', include_docs_urls(title=API_TITLE, description=API_DESCRIPTION))
]
  • 通过get_resolver方法，完成了对settings.ROOT_URLCONF关联的整个路由系统的加载，返回一个URLResolver实例；
  • urlpatterns其实就是一个URLPattern和URLResolver的集合列表，包含inclue的为URLResolver，直接跟着一个view方法的则是URLPattern；

django.urls.conf._path

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def _path(route, view, kwargs=None, name=None, Pattern=None):
    if isinstance(view, (list, tuple)):
        # For include(...) processing.
        pattern = Pattern(route, is_endpoint=False)
        urlconf_module, app_name, namespace = view
        return URLResolver(
            pattern,
            urlconf_module,
            kwargs,
            app_name=app_name,
            namespace=namespace,
        )
    elif callable(view):
        pattern = Pattern(route, name=name, is_endpoint=True)
        return URLPattern(pattern, view, kwargs, name)
    else:
        raise TypeError('view must be a callable or a list/tuple in the case of include().')
  • 这个方法其实就是url(r'^schema/$', schema_view)追溯过去的，解释了上面的问题：什么情况是URLPattern和什么情况下是URLResolver；
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class URLPattern:
    def __init__(self, pattern, callback, default_args=None, name=None):
        self.pattern = pattern
        self.callback = callback  # the view
        self.default_args = default_args or {}
        self.name = name

    def __repr__(self):
        return '<%s %s>' % (self.__class__.__name__, self.pattern.describe())
      
  	########省 略################################
      
    def resolve(self, path):  # 负责完成path的匹配，并返回匹配的视图方法；
        match = self.pattern.match(path)
        if match:
            new_path, args, kwargs = match
            # Pass any extra_kwargs as **kwargs.
            kwargs.update(self.default_args)
            return ResolverMatch(self.callback, args, kwargs, self.pattern.name)
  • 每个URLPattern都需要指定如下几个内容：
    • 一个正则表达式字符串。
    • 一个可调用对象，通常为一个视图函数或一个指定视图函数路径的字符串。
    • 可选的要传递给视图函数的默认参数（字典形式）。
    • 一个可选的name参数。
  • 如果有include，则递归生成上面的模式；
  • resolve(self, path)负责完成path的匹配，并返回匹配的视图方法；
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
class URLResolver:
    def __init__(self, pattern, urlconf_name, default_kwargs=None, app_name=None, namespace=None):
        self.pattern = pattern
        # urlconf_name is the dotted Python path to the module defining
        # urlpatterns. It may also be an object with an urlpatterns attribute
        # or urlpatterns itself.
        self.urlconf_name = urlconf_name
        self.callback = None
        self.default_kwargs = default_kwargs or {}
        self.namespace = namespace
        self.app_name = app_name
        self._reverse_dict = {}
        self._namespace_dict = {}
        self._app_dict = {}
        # set of dotted paths to all functions and classes that are used in
        # urlpatterns
        self._callback_strs = set()
        self._populated = False
        self._local = threading.local()
    
    ################################省 略################################
    
   def resolve(self, path):
        path = str(path)  # path may be a reverse_lazy object
        tried = []
        match = self.pattern.match(path)
        if match:
            new_path, args, kwargs = match
            for pattern in self.url_patterns:
                try:
                    sub_match = pattern.resolve(new_path)
                except Resolver404 as e:
                    sub_tried = e.args[0].get('tried')
                    if sub_tried is not None:
                        tried.extend([pattern] + t for t in sub_tried)
                    else:
                        tried.append([pattern])
                else:
                    if sub_match:
                        # Merge captured arguments in match with submatch
                        sub_match_dict = {**kwargs, **self.default_kwargs}
                        # Update the sub_match_dict with the kwargs from the sub_match.
                        sub_match_dict.update(sub_match.kwargs)
                        # If there are *any* named groups, ignore all non-named groups.
                        # Otherwise, pass all non-named arguments as positional arguments.
                        sub_match_args = sub_match.args
                        if not sub_match_dict:
                            sub_match_args = args + sub_match.args
                        return ResolverMatch(
                            sub_match.func,
                            sub_match_args,
                            sub_match_dict,
                            sub_match.url_name,
                            [self.app_name] + sub_match.app_names,
                            [self.namespace] + sub_match.namespaces,
                        )
                    tried.append([pattern])
            raise Resolver404({'tried': tried, 'path': new_path})
        raise Resolver404({'path': path})

    @cached_property
    def urlconf_module(self):
        if isinstance(self.urlconf_name, str):
            return import_module(self.urlconf_name)
        else:
            return self.urlconf_name

    @cached_property
    def url_patterns(self):
        # urlconf_module might be a valid set of patterns, so we default to it
        patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
        try:
            iter(patterns)
        except TypeError:
            msg = (
                "The included URLconf '{name}' does not appear to have any "
                "patterns in it. If you see valid patterns in the file then "
                "the issue is probably caused by a circular import."
            )
            raise ImproperlyConfigured(msg.format(name=self.urlconf_name))
        return patterns

  • 一个URL Resolve可以包含多个URL Pattern，也可以包含多个其他URL Resolve。 通过这种包含结构设计，实现Django对URL的层级解析，也是Django实现app与项目解耦的关键。通常由include方法操作的URL配置模块，最终会被解释成为URL分解器。

  • 每个URL分解器都包含两个重要的变量:

    • 一个正则表达式字符串。URL开始部分是否匹配正则表达式，如匹配，去除成功匹配部分后余下部分匹配包含的URL模式和URL分解器。
    • URL配置模块名或URL配置模块的引用。
  • resolve(self, path)在这个方法开始遍历URLResolve，直到获得正确匹配的URLPattern，并调用其resolve方法
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
class ResolverMatch:
    def __init__(self, func, args, kwargs, url_name=None, app_names=None, namespaces=None):
        self.func = func
        self.args = args
        self.kwargs = kwargs
        self.url_name = url_name

        # If a URLRegexResolver doesn't have a namespace or app_name, it passes
        # in an empty value.
        self.app_names = [x for x in app_names if x] if app_names else []
        self.app_name = ':'.join(self.app_names)
        self.namespaces = [x for x in namespaces if x] if namespaces else []
        self.namespace = ':'.join(self.namespaces)

        if not hasattr(func, '__name__'):
            # A class-based view
            self._func_path = func.__class__.__module__ + '.' + func.__class__.__name__
        else:
            # A function-based view
            self._func_path = func.__module__ + '.' + func.__name__

        view_path = url_name or self._func_path
        self.view_name = ':'.join(self.namespaces + [view_path])

    def __getitem__(self, index):
        return (self.func, self.args, self.kwargs)[index]

    def __repr__(self):
        return "ResolverMatch(func=%s, args=%s, kwargs=%s, url_name=%s, app_names=%s, namespaces=%s)" % (
            self._func_path, self.args, self.kwargs, self.url_name,
            self.app_names, self.namespaces,
        )匹配结果（Resolver Match）

  • 匹配结果是指当URL被正确匹配时，需返回的匹配结果，匹配结果需指定以下几个内容:

    • 一个可调用对象。通常是视图函数
    • 视图函数参数。通常是URL模式中正则表达式命名组匹配的值
    • 视图函数关键字参数。通常是url方法中设置传递给视图函数的参数(字典形式)
    • 可选的URL名称参数。
    • 可选的APP名称参数。
    • 可选的命名空间参数。

  • 用来表示匹配结果。ResolverMatch类实现了__getitem__方法，可以同元组操作一样,获取视图函数引用与视图函数参数，从而具备调用视图函数的条件。

3、总结

上面所有的过程都是Django的服务启动后，一个新的request达到后，经过怎么的流程到达对应view方法的步骤，简单做一个总结；

  1. 在启动Django服务的时候，会调用load_middleware来加载所有的中间键到BaseHandler的_middleware_chain变量中，可以等同一个多层的装饰器；
  2. 当一个新的请求到达后，会调用WSGIHandler的__call__方法，最终执行到BaseHandler中的get_reponse方法；
  3. 在get_reponse方法中调用了_middleware_chain方法，即依次倒序执行中间件中的方法，最后执行到_get_response方法；
  4. 在这个方法，request对应的视图方法被匹配到，执行并返回结果，即URLResolver根据path info匹配到正确的ResolverMatch，然后调用其中的view方法，并返回respone。

参考：Django 源码学习（2）——url路由