How to Change POS Configuration
POS provides a lot of knobs so that,
1) The benefits of using fast NVMe-oF storage could be fully achieved with various environments and 2) The integration efforts with existing infrastructure could be minimized.
In this section, we will go over the list of configuration variables that can change the behavior of POS. Some of those are meaningful to POS developers only, unlikely to change by users. Though, we explicitly list them up here to share hidden assumptions and magic numbers to be able to diagnose any side effects quickly that could happen due to user-specific environment issues.
The configuration is encoded in json at /etc/pos/pos.conf. You can create one by yourself or just copy one from the ~/config directory. There is one more way to create a pos.conf file under /etc/pos. After you build the POS by the script or just by issuing “make” from the root of POS directory, issue “make install” then the binaries built will be installed to the install directory(usually under /usr/local/bin) and ~/config/pos.conf will be copied to /etc/pos/pos.conf as well. In case there is already one under /etc/pos, you need to delete it, first and do “make install”. This should be done like this since there could be a case that the pos.conf which was already under /etc/pos is accidentally substituted without intent.
The following tables describe a list of (config name, the range of valid values, the default value, whether it can change at runtime, whether it is supported today). </br>
| Name | Category | Value | Note |
|---|---|---|---|
| journal.enable | Range of valid values | true | false | If true, POS logs metadata updates and performs recovery upon power-failure or crash. |
| Default value | true | ||
| Changeable at runtime | NO | ||
| Current Support Status | Experimental | ||
| Name | Category | Value | Note |
| logger. logfile_size_in_mb | Range of valid values | 1 ~ UINT32_MAX | If the size of the log becomes larger than the config, POS rotates the log by renaming the current log file and opening a new one. |
| Default value | 50 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| logger. logfile_rotation_count | Range of valid values | 1 ~ UINT32_MAX | The number of rotated log files that POS keeps |
| Default value | 20 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| logger. min_allowable_log_level | Range of valid values | off | debug | info | trace | warn | err | critical | Any log level equal to or higher than this config value would be captured by POS logger. |
| Default value | debug | ||
| Changeable at runtime | YES | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| logger. deduplication_enabled | Range of valid values | true | false | If true, POS omits duplicated logs. |
| Default value | TRUE | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| logger. deduplication_ sensitivity_in_msec | Range of valid values | 1 ~ UINT32_MAX | Time(in milliseconds) to determine log duplication when “deduplication_enabled” is true |
| Default value | 20 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| ioat.enable | Range of valid values | true | false | If true, POS uses H/W DMA engine supported by Intel XEON CPU and Intel Chipset to accelerate Internal memory copy. Otherwise POS copies them using CPU. |
| Default value | TRUE | If the system doesn’t support Intel I/OAT, POS automatically use CPU for internal memory copy. So this is just to disable I/OAT feature on Intel I/OAT supported system. | |
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| ioat.ioat_cnt_numa0 | Range of valid values | 0 ~ (# of cores - “ioat_cnt_numa1”) | It specifies how many IOAT H/Ws on NUMA socket 0 to be used. Generally, each CPU has 8 independent IOAT H/Ws. |
| Default value | 8 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| ioat.ioat_cnt_numa1 | Range of valid values | 0 ~ (# of cores - “ioat_cnt_numa0”) | It specifies how many IOAT H/Ws on NUMA socket 1 to be used. Generally, each CPU has 8 independent IOAT H/Ws. |
| Default value | 8 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager. use_config | Range of valid values | true | false | If true, POS reads the following configs from config file: “reactor, udd_io_worker, event_scheduler, event_worker, general_usage, qos, meta_scheduler, meta_io”. Otherwise(including affinity_manager object is omitted), they are,set as their default values. |
| Default value | TRUE | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager. reactor | Range of valid values | core # | It specifies the CPU core ID(s) to run reactor thread(s) on. The valid value can be combination of below formats, “0”: Just a single CPU core #0 “2,7”: Two CPU cores which are CPU #2 and CPU,#7, separated by comma(s) “1-8”: Eight CPU cores which are 1, 2, 3, 4, 5,,6, 7, 8, separated by dash. Reactor is mainly a frontend IO thread which handles incoming IO,requests from an initiator. |
| Default value | 0 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager. udd_io_worker | Range of valid values | core # | It specifies CPU core ID(s) to run IO Worker thread(s) on. The valid value can be combination of below formats, “0”: Just a single CPU core #0 “2,7”: Two CPU cores which are CPU #2 and CPU,#7, separated by comma(s) “1-8”: Eight CPU cores which are 1, 2, 3, 4, 5,,6, 7, 8, separated by dash. IO Worker is a backend IO thread which handles internal IO,requests inside of POS to the backend storage devices. |
| Default value | 1 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager. event_scheduler | Range of valid values | core # | It specifies the CPU core ID to run the Event Scheduler thread on. The valid value is in below format, “0”: Just a single CPU core #0 Event Scheduler dispatches internal Events to particular Event,Worker to serve each Event. |
| Default value | 2 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager. event_worker | Range of valid values | core # | It specifies the CPU core ID(s) to run Event Worker thread(s) on. The valid value can be combination of below formats, “0”: Just a single CPU core #0, “2,7”: Two CPU cores which are CPU #2 and CPU,#7, separated by comma(s), “1-8”: Eight CPU cores which are 1, 2, 3, 4, 5,,6, 7, 8, separated by dash. Event Worker runs each Event dispatched from Event Scheduler. |
| Default value | 3-5 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager. general_usage | Range of valid values | core # | It specifies CPU core ID(s) to run General thread(s) on. The valid value can be combination of below formats, “0”: Just a single CPU core #0, “2,7”: Two CPU cores which are CPU #2 and CPU,#7, separated by comma(s) “1-8”: Eight CPU cores which are 1, 2, 3, 4, 5,,6, 7, 8, separated by dash. General core runs threads including CLI Server, CLI requests,,Device monitor, and etc. |
| Default value | 6 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager.qos | Range of valid values | core # | It specifies the CPU core ID to run QoS thread on. The valid value is in below format,, “0”: Just a single CPU core #0 QoS thread mainly controls the bandwidth of each Internal Event related to IO Service (Userdata flush, Rebuild, GC). |
| Default value | 7 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager. meta_scheduler | Range of valid values | core # | It specifies CPU core ID to run Meta Scheduler thread on. The valid value is in below format, “0”: Just a single CPU core #0 Meta Scheduler dispatches internal request message to particular,Meta IO thread to update or read metadata. |
| Default value | 8 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| affinity_manager. meta_io | Range of valid values | core # | It specifies CPU core ID(s) to run Meta IO thread(s) on., The valid value is in below format, “0”: Just a single CPU core #0 “2,7”: Two CPU cores which are CPU #2 and CPU,#7, separated by comma(s) “1-8”: Eight CPU cores which are 1, 2, 3, 4, 5,,6, 7, 8, separated by dash., Meta IO thread runs each Request it gets from Meta Scheduler. |
| Default value | 9-10 | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | This is used for POS developers only. Do not change the value for any reason. | |
| Name | Category | Value | Note |
| user_nvme_driver. use_config | Range of valid values | true | false | If true, POS reads the following configs from config file and overrides: “ssd_timeout_us, retry_count_backend_io, retry_count_frontend_io”. Otherwise, the hard-coded default values are used. |
| Default value | FALSE | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| user_nvme_driver. ssd_timeout_us | Range of valid values | 0 ~ 100000000 (100 secs.) | It specifies timeout value in microseconds for each backend storage device |
| Default value | 8000000 (8 secs.) | It defaults to 80000000 (80 secs.) if “use_config” is false. | |
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| user_nvme_driver. retry_count_backend_io | Range of valid values | 0 ~ 10 | It specifies how many times to retry an I/O request on failure from backend IO thread to a device including backend storage devices and write buffer. |
| Default value | 5 | It defaults to 0 if “use_config” is false. | |
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| user_nvme_driver. retry_count_frontend_io | Range of valid values | 0 ~ 10 | It specifies how many times to retry an IO request on failure from frontend IO thread to a device including backend storage devices and write buffer. |
| Default value | 3 | It defaults to 0 if “use_config” is false. | |
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
| Name | Category | Value | Note |
| perf_impact.rebuild | Range of valid values | “low” | “medium” | “high” | It specifies the priority of rebuild bandwidth compared to user IO. “high” is the highest priority to operate which is the, same as user IO so that rebuild and user IO will compete for the bandwidth. “low” is the lowest priority to operate so that rebuild,will get the minimum bandwidth. |
| Default value | “low” | ||
| Changeable at runtime | NO | ||
| Current Support Status | Supported | ||
