Validation pass when object, list or integer returned instead of string for object string property (must fail)

[andyceo]

Hi!

We faced with following bug. Some object's property declared as string, If this object's property returned by API is object itself, or integer, or list, no errors generated.

If API returns wrong property type for example int property, InvalidMediaTypeValue thrown (as expected).

Tested on version 0.5.0, 0.7.1, master - same results.

Expected behavior

InvalidMediaType should be thrown

Steps to reproduce

1. Install openapi-core:

    pip3 install openapi-core
   

2.. Create and save following fatless specification as test.yml:

openapi: "3.0.1"

info:
  version: "0.1"
  title: Object instead of string
  description: Test for if returns objects instead of string

components:
  schemas:
	SomeObject:
	  type: object
	  properties:
		someprop:
		  description: Some property
		  type: string
		someint:
		  type: integer

paths:
  /getsomeobject:
	get:
	  summary: Get the SomeObject
	  operationId: getSomeObject
	  responses:
		'200':
		  description: This is SomeObject
		  content:
			application/json:
			  schema:
				$ref: '#/components/schemas/SomeObject'

2. Create and save following script as test.py:

    #!/usr/bin/env python3
    
    # -*- coding: utf-8 -*-
    import json
    import sys
    import yaml
    from openapi_core import create_spec
    from openapi_core.shortcuts import RequestValidator, ResponseValidator
    from openapi_core.wrappers.mock import MockRequest, MockResponse
    
    
    def validate(openapi_file):
        with open(openapi_file, 'r') as myfile:
            spec_dict = yaml.safe_load(myfile)
            spec = create_spec(spec_dict)
    
            openapi_request = MockRequest('localhost', 'get', '/getsomeobject')
            validator = RequestValidator(spec)
            result = validator.validate(openapi_request)
            request_errors = result.errors
    
            # PASS (must PASS)
            data = json.dumps({
                'someprop': 'content'
            })
    
            # PASS (must FAIL)
            data = json.dumps({
                'someprop': {
                    'nested_object_property': 'content',
                    'nested_object_another property': 13,
                }
            })
    
            # PASS (must FAIL)
            data = json.dumps({
                'someprop': ['dfdfdf', 'dfdfdfsssss']
            })
    
            # PASS (must FAIL)
            data = json.dumps({
                'someprop': 123
            })
    
            # PASS (must FAIL)
            data = json.dumps({
                'someprop': 123
            })
    
            # FAIL (must FAIL)
            data = json.dumps({
                'someint': 'content'
            })
    
            # FAIL (must FAIL)
            data = json.dumps({
                'someprop': 'dsdsd',
                'someint': 123,
                'not_in_scheme_prop': 123
            })
    
            openapi_response = MockResponse(data)
            validator = ResponseValidator(spec)
            result = validator.validate(openapi_request, openapi_response)
            response_errors = result.errors
    
            print('Request errors: {} Response errors: {}'.format(request_errors, response_errors))
    
    
    if __name__ == "__main__":
        if len(sys.argv) < 2:
            print("Specify path to openapi.yaml file!")
            exit(1)
        else:
            validate(sys.argv[1])
   

3. Execute script to validate spec:

    python3 test.py test.yml
   

4. Try to comment out test payloads to see actual results.

[andyceo]

Initial report is here: ergoplatform/ergo#555

[danielbradburn]

I have recently started experimenting with openapi_core and also came across this issue. I think the problem is that when unmarshalling a string and no format is given str is used, which in python means lot's of things can correctly be unmarshalled as strings (including numbers, arrays, objects etc.)

I have worked around this issue for now by monkey patching so that I can continue experimenting with this really cool library (thanks for this by the way @p1c2u !)

@contextlib.contextmanager
def strict_str():
    """
    openapi_core unmarshals and validates strings by converting whatever
    is in the response to a str, and then validating that what we get is
    a str. This is of course means that lot's of things convert
    fine to a string. This means that we cannot validate string
    properties.

    To workaround this issue this function provides a means to patch
    openapi_core to use our own custom formatting for strings which
    strictly checks that a value is a string.

    For example...

      >>> with strict_str():
      ...   validator = ResponseValidator(...)
    """

    def strict_to_str(x):
        if not isinstance(x, str):
            raise OpenAPISchemaError(f"Expected str but got {type(x)} -- {x}")
        return x

    original = Schema.STRING_FORMAT_CALLABLE_GETTER
    patched = dict(original)
    patched[SchemaFormat.NONE] = Format(strict_to_str, lambda x: isinstance(x, str))

    target = "openapi_core.schema.schemas.models.Schema.STRING_FORMAT_CALLABLE_GETTER"
    with patch(target, patched):
        yield

@p1c2u what do you think about making the unmarshalling of strings more strict so that this patching is not needed? I would be happy to make a PR if you agree with the approach (basically when no format is given we use strict type checking to ensure we actually have a string)

[danielbradburn]

Ah actually I see there is already a PR!

[andyceo]

Oh thank you! I'll check this within 2-3 weeks.