{"meta":{"title":"Extractor options","intro":"Control how the CodeQL CLI builds databases for analysis with extractor options.","product":"Security and code quality","breadcrumbs":[{"href":"/en/code-security","title":"Security and code quality"},{"href":"/en/code-security/reference","title":"Reference"},{"href":"/en/code-security/reference/code-scanning","title":"Code scanning"},{"href":"/en/code-security/reference/code-scanning/codeql","title":"CodeQL"},{"href":"/en/code-security/reference/code-scanning/codeql/codeql-cli","title":"CodeQL CLI"},{"href":"/en/code-security/reference/code-scanning/codeql/codeql-cli/extractor-options","title":"Extractor options"}],"documentType":"article"},"body":"# Extractor options\n\nControl how the CodeQL CLI builds databases for analysis with extractor options.\n\n<!--The CodeQL CLI man pages include a link to this article. If you rename this article,\nmake sure that you also update the MS short link: https://aka.ms/codeql-cli-docs/extractor-options.-->\n\n## Available extractor options\n\nEach extractor defines its own set of configuration options for building a queryable CodeQL database from source code. To find out which options are available for a particular extractor, you can run either of the following commands:\n* `codeql resolve languages --format=betterjson`\n* `codeql resolve extractor --language=LANGUAGE --format=betterjson`\n\nThe `betterjson` output format provides the root paths of extractors and additional information. The output of `codeql resolve extractor --language=LANGUAGE --format=betterjson` will often be formatted like the following example:\n\n```json\n{\n    \"extractor_root\" : \"/home/user/codeql/java\",\n    \"extractor_options\" : {\n        \"option1\" : {\n            \"title\" : \"Java extractor option 1\",\n            \"description\" : \"An example string option for the Java extractor.\",\n            \"type\" : \"string\",\n            \"pattern\" : \"[a-z]+\"\n        },\n        \"group1\" : {\n            \"title\" : \"Java extractor group 1\",\n            \"description\" : \"An example option group for the Java extractor.\",\n            \"type\" : \"object\",\n            \"properties\" : {\n                \"option2\" : {\n                    \"title\" : \"Java extractor option 2\",\n                    \"description\" : \"An example array option for the Java extractor\",\n                    \"type\" : \"array\",\n                    \"pattern\" : \"[1-9][0-9]*\"\n                }\n            }\n        }\n    }\n}\n```\n\nThe extractor option names and descriptions are listed under `extractor_options`. Each option may contain the following fields:\n\n* `title` (required): The title of the option\n* `description` (required): The description of the option\n* `type` (required): The type of the option, which can be\n  * `string`: indicating that the option can have a single string value\n  * `array`: indicating that the option can have a sequence of string values\n  * `object`: indicating that it is not an option itself, but a grouping that may contain other options and option groups\n* `pattern` (optional): The regular expression patterns that all values of the option should match. Note that the extractor may impose additional constraints on option values that are not or cannot be expressed in this regular expression pattern. Such constraints, if they exist, would be explained under the description field.\n* `properties` (optional): A map from extractor option names in the option group to the corresponding extractor option descriptions. This field can only be present for option groups. For example, options of `object` type.\n\nIn the example above, the extractor declares two options:\n\n* `option1` is a `string` option with value matching `[a-z]+`\n* `group1.option2` is an `array` option with values matching `[1-9][0-9]\\*`\n\n## Commands for setting extractor options\n\nThe CodeQL CLI supports setting extractor options in subcommands that directly or indirectly invoke extractors. These commands are:\n\n* `codeql database create`\n* `codeql database start-tracing`\n* `codeql database trace-command`\n* `codeql database index-files`\n\nWhen running these subcommands, you can set extractor options with the `--extractor-option` CLI option. For example:\n\n* `codeql database create --extractor-option java.option1=abc ...`\n* `codeql database start-tracing --extractor-option java.group1.option2=102 ...`\n\n`--extractor-option` requires exactly one argument of the form `extractor_option_name=extractor_option_value`:\n* `extractor_option_name` is the name of the extractor (in this example, `java`) followed by a period and then the name of the extractor option (in this example, either `option1` or `group1.option2`).\n* `extractor_option_value` is the value being assigned to the extractor option. The value must match the regular expression pattern of the extractor option (if it exists), and it must not contain newline characters.\n\nUsing `--extractor-option` to assign an extractor option that does not exist is an error.\n\nThe CodeQL CLI accepts multiple `--extractor-option` options in the same invocation. If you set a `string` extractor option multiple times, the last option value overwrites all previous ones. If you set an array extractor option multiple times, all option values are concatenated in order.\n\nYou can also specify extractor option names without the extractor name. For example:\n\n* `codeql database create --extractor-option option1=abc ...`\n* `codeql database start-tracing --extractor-option group1.option2=102 ...`\n\nIf you do not specify an extractor name, the extractor option settings will apply to all extractors that declare an option with the given name. In the above example, the first command would set the extractor option `option1` to `abc` for the `java` extractor and every extractor that has an option of `option1`, for example the `cpp` extractor, if the `option1` extractor option exists for that extractor.\n\n## File format for extractor options\n\nYou can also set extractor options through a file. The CodeQL CLI subcommands that accept `--extractor-option` also accept `--extractor-options-file`, which has a required argument of the path to a YAML file (with extension `.yaml` or `.yml`) or a JSON file (with extension `.json`). For example:\n\n* `codeql database create --extractor-options-file options.yml ...`\n* `codeql database start-tracing --extractor-options-file options.json ...`\n\nEach option file contains a tree structure of nested maps. At the root is an extractor map key, and beneath it are map keys that correspond to extractor names. Starting at the third level, there are extractor options and option groups.\n\nIn JSON:\n\n```json\n{\n     \"extractor\" : {\n        \"java\": {\n            \"option1\" : \"abc\",\n            \"group1\" : {\n                \"option2\" : [ 102 ]\n            }\n        }\n    }\n}\n```\n\nIn YAML:\n\n```yaml\nextractor:\n    java:\n        option1: \"abc\"\n        group1:\n            option2: [ 102 ]\n```\n\nThe value for a `string` extractor option must be a string or a number (which will be converted to a string before further processing).\n\nThe value for an `array` extractor option must be an array of strings or numbers.\n\nThe value for an option group (of type `object`) must be a map, which may contain nested extractor options and option groups.\n\nEach extractor option value must match the regular expression pattern of the extractor option (if it exists), and it must not contain newline characters.\n\nAssigning an extractor option that does not exist is an error. You can make the CodeQL CLI ignore unknown extractor options by using a special `__allow_unknown_properties` Boolean field. For example, the following option file asks the CodeQL CLI to ignore all unknown extractor options and option groups under `group1`:\n\n```yaml\nextractor:\n    java:\n        option1: \"abc\"\n        group1:\n            __allow_unknown_properties: true\n            option2: [ 102 ]\n```\n\nYou can specify `--extractor-options-file` multiple times. The extractor option assignments are processed in the following order:\n\n1. All extractor option files specified by `--extractor-options-file` are processed in the order they appear on the command line, then\n1. All extractor option assignments specified by `--extractor-option` are processed in the order they appear on the command line\n\nThe same rules govern what happens when the same extractor option is set multiple times, regardless of whether the assignments are done using `--extractor-option`, using `--extractor-options-file`, or some combination of the two. If you set a `string` extractor option multiple times, the last option value overwrites all previous values. If you set an `array` extractor option multiple times, all option values are concatenated in order."}