OpenStack 源码系列之 Keystone：API 路由

作为一个服务，Keystone 跟其他模块一样向外提供了 Rest API，以供其他 HTTP 客户端（如 Keystoneclient、POSTMan、Curl）调用。这些 Rest API 调用并不是直接调用底层的 API 来来实现，而是通过路由，转发到底层的 API 去执行并返回结果。只要理解了一个 Rest API 的路由过程，就可以以此类推知道其他 Rest API 是如何路由的。下文我们以用户创建（超级管理员权限）为例说明该过程。

1. 从 PasteDeploy 配置文件确定 API 入口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# etc/keystone-paste.ini

[app:admin_service]
use = egg:keystone#admin_service

......

[pipeline:admin_api]
# The last item in this pipeline must be admin_service or an equivalent
# application. It cannot be a filter.
pipeline = ... admin_service

......

[composite:admin]
use = egg:Paste#urlmap
/v2.0 = admin_api
...

我们会发现超级管理员 API 的入口在 egg:keystone#admin_service。但这个入口并没有具体指向哪个函数，所以我们需要进一步确定该入口的具体指向。

2. 从 setup.cfg 确定具体服务入口

1
2
3
4
5
# setup.cfg

paste.app_factory =
    admin_service = keystone.version.service:admin_app_factory
    ...

3. 确定具体的 Controller 类
一旦确定了服务入口，要确定具体的资源 Controller 类是很简单的：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
# keystone/version/service.py

@fail_gracefully
@warn_local_conf
def admin_app_factory(global_conf, **local_conf):
    controllers.register_version('v2.0')
    return wsgi.ComposingRouter(routes.Mapper(),
                                [identity_routers.Admin(),
                                 assignment_routers.Admin(),
                                 token_routers.Router(),
                                 resource_routers.Admin(),
                                 admin_crud.Router(),	# 集合了各种资源的 CRUD 操作
                                 routers.VersionV2('admin'),
                                 routers.Extension()])


# keystone/v2_crud/admin_crud.py

class Router(wsgi.ComposableRouter):
    """Previously known as the OS-KSADM extension.

    Provides a bunch of CRUD operations for internal data types.

    """

    def add_routes(self, mapper):
        ......
        user_controller = identity.controllers.User()
        ......
        # User Operations
        mapper.connect(
            '/users',
            controller=user_controller,
            action='get_users',
            conditions=dict(method=['GET']))
        mapper.connect(
            '/users',
            controller=user_controller,
            action='create_user',	# Controller 中 API 的名称（动作）
            conditions=dict(method=['POST']))
        # NOTE(termie): not in diablo
        mapper.connect(
            '/users/{user_id}',
            controller=user_controller,
            action='update_user',
            conditions=dict(method=['PUT']))
        mapper.connect(
            '/users/{user_id}',
            controller=user_controller,
            action='delete_user',
            conditions=dict(method=['DELETE']))

到这一步大家是不是就很熟悉了？关于 Routes 模块如何工作可以参考之前博文 Routes。

4. 定位到 Controller 具体的 API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
# keystone/identity/controllers.py

@dependency.requires('assignment_api', 'identity_api', 'resource_api')
class User(controller.V2Controller):

    @controller.v2_deprecated
    def get_user(self, request, user_id):
        ......

    @controller.v2_deprecated
    def get_users(self, request):
         ......

    @controller.v2_deprecated
    def get_user_by_name(self, request, user_name):
        ......

    # CRUD extension
    @controller.v2_deprecated
    def create_user(self, request, user):
                validation.lazy_validate(schema.user_create_v2, user)
        user = self._normalize_OSKSADM_password_on_request(user)
        user = self.normalize_username_in_request(user)
        user = self._normalize_dict(user)
        self.assert_admin(request)

        default_project_id = user.pop('tenantId', None)
        if default_project_id is not None:
            # Check to see if the project is valid before moving on.
            self.resource_api.get_project(default_project_id)
            user['default_project_id'] = default_project_id

        self.resource_api.ensure_default_domain_exists()

        # The manager layer will generate the unique ID for users
        user_ref = self._normalize_domain_id(request, user.copy())
        new_user_ref = self.v3_to_v2_user(
            self.identity_api.create_user(
                user_ref, initiator=request.audit_initiator
            )
        )

        if default_project_id is not None:
            self.assignment_api.add_user_to_project(default_project_id,
                                                    new_user_ref['id'])
        return {'user': new_user_ref}

    @controller.v2_deprecated
    def update_user(self, request, user_id, user):
        ......

    @controller.v2_deprecated
    def delete_user(self, request, user_id):
        ......


# keystone/common/controller.py

class V2Controller(wsgi.Application):
    ......


# keystone/common/wsgi.py

@dependency.requires('assignment_api', 'policy_api', 'token_provider_api')
class Application(BaseApplication):

    @webob.dec.wsgify(RequestClass=request_mod.Request)
    def __call__(self, req):
        arg_dict = req.environ['wsgiorg.routing_args'][1]
        action = arg_dict.pop('action')
        del arg_dict['controller']

        params = req.environ.get(PARAMS_ENV, {})
        params.update(arg_dict)

        # TODO(termie): do some basic normalization on methods
        method = getattr(self, action)

        # NOTE(morganfainberg): use the request method to normalize the
        # response code between GET and HEAD requests. The HTTP status should
        # be the same.
        LOG.info('%(req_method)s %(uri)s', {
            'req_method': req.method.upper(),
            'uri': wsgiref.util.request_uri(req.environ),
        })

        params = self._normalize_dict(params)

        try:
            result = method(req, **params)
        except exception.Unauthorized as e:
            LOG.warning(
                _LW("Authorization failed. %(exception)s from "
                    "%(remote_addr)s"),
                {'exception': e, 'remote_addr': req.environ['REMOTE_ADDR']})
            return render_exception(e,
                                    context=req.context_dict,
                                    user_locale=best_match_language(req))
        except exception.Error as e:
            LOG.warning(six.text_type(e))
            return render_exception(e,
                                    context=req.context_dict,
                                    user_locale=best_match_language(req))
        except TypeError as e:
            LOG.exception(six.text_type(e))
            return render_exception(exception.ValidationError(e),
                                    context=req.context_dict,
                                    user_locale=best_match_language(req))
        except Exception as e:
            LOG.exception(six.text_type(e))
            return render_exception(exception.UnexpectedError(exception=e),
                                    context=req.context_dict,
                                    user_locale=best_match_language(req))

        if result is None:
            return render_response(
                status=(http_client.NO_CONTENT,
                        http_client.responses[http_client.NO_CONTENT]))
        elif isinstance(result, six.string_types):
            return result
        elif isinstance(result, webob.Response):
            return result
        elif isinstance(result, webob.exc.WSGIHTTPException):
            return result

        response_code = self._get_response_code(req)
        return render_response(body=result,
                               status=response_code,
                               method=req.method)

    ......

5. 定位到数据库操作
最终的创建操作要反映到数据库：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
# keystone/identity/core.py

class Manager(manager.Manager):
    ......

    @domains_configured
    @exception_translated('user')
    def create_user(self, user_ref, initiator=None):
        user = user_ref.copy()
        if 'password' in user:
            validators.validate_password(user['password'])
        user['name'] = clean.user_name(user['name'])
        user.setdefault('enabled', True)
        user['enabled'] = clean.user_enabled(user['enabled'])
        domain_id = user['domain_id']
        self.resource_api.get_domain(domain_id)

        self._assert_default_project_id_is_not_domain(
            user_ref.get('default_project_id'))

        # For creating a user, the domain is in the object itself
        domain_id = user_ref['domain_id']
        driver = self._select_identity_driver(domain_id)
        user = self._clear_domain_id_if_domain_unaware(driver, user)
        # Generate a local ID - in the future this might become a function of
        # the underlying driver so that it could conform to rules set down by
        # that particular driver type.
        user['id'] = uuid.uuid4().hex
        ref = driver.create_user(user['id'], user)
        notifications.Audit.created(self._USER, user['id'], initiator)
        return self._set_domain_id_and_mapping(
            ref, domain_id, driver, mapping.EntityType.USER)


# keystone/identity/backends/sql.py

class Identity(base.IdentityDriverBase):
    ......

    @sql.handle_conflicts(conflict_type='user')
    def create_user(self, user_id, user):
        user = utils.hash_user_password(user)
        with sql.session_for_write() as session:
            user_ref = model.User.from_dict(user)
            if self._change_password_required(user_ref):
                user_ref.password_ref.expires_at = datetime.datetime.utcnow()
            user_ref.created_at = datetime.datetime.utcnow()
            session.add(user_ref)
            # Set resource options passed on creation
            resource_options.resource_options_ref_to_mapper(
                user_ref, model.UserOption)
            return base.filter_user(user_ref.to_dict())


# keystone/identity/backends/sql_model.py

class User(sql.ModelBase, sql.DictBase):
    __tablename__ = 'user'
    attributes = ['id', 'name', 'domain_id', 'password', 'enabled',
                  'default_project_id', 'password_expires_at']
    readonly_attributes = ['id', 'password_expires_at']
    resource_options_registry = iro.USER_OPTIONS_REGISTRY
    id = sql.Column(sql.String(64), primary_key=True)
    domain_id = sql.Column(sql.String(64), nullable=False)
    _enabled = sql.Column('enabled', sql.Boolean)
    extra = sql.Column(sql.JsonBlob())
    default_project_id = sql.Column(sql.String(64))
    _resource_option_mapper = orm.relationship(
        'UserOption',
        single_parent=True,
        cascade='all,delete,delete-orphan',
        lazy='subquery',
        backref='user',
        collection_class=collections.attribute_mapped_collection('option_id'))
    local_user = orm.relationship('LocalUser', uselist=False,
                                  single_parent=True, lazy='subquery',
                                  cascade='all,delete-orphan', backref='user')
    federated_users = orm.relationship('FederatedUser',
                                       single_parent=True,
                                       lazy='subquery',
                                       cascade='all,delete-orphan',
                                       backref='user')
    nonlocal_user = orm.relationship('NonLocalUser',
                                     uselist=False,
                                     single_parent=True,
                                     lazy='subquery',
                                     cascade='all,delete-orphan',
                                     backref='user')
    created_at = sql.Column(sql.DateTime, nullable=True)
    last_active_at = sql.Column(sql.Date, nullable=True)
    ......

至此，一个完整的 API 调用就算完成了！

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

谢宏峰，毕业于暨南大学（2009 - 2013）、中国科学院（2013 - 2016），现就职于深信服科技，从事 OpenStack 开发。