These functions are used in local
Change-Id: Ia56d70e2ff69295a31b5ac411dcfdcdc854bda18
Signed-off-by: Kasin Li <donglil@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Update the logic that checks maximum allowed power level to
accommodate new frequency corners.
Change-Id: I85c9db7e39c75b14ee77464ca6fcdf5855653b25
Signed-off-by: Mohammed Mirza Mandayappurath Manzoor <mmandaya@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Move devfreq_devbw to late init call to defer the probing.
Change-Id: I5eecb8f4c3e177696f8c4ebd04b84daef6fc65c2
Signed-off-by: Santosh Mardi <gsantosh@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
devfreq_qcom_qoslat platform driver does not
support the manual bind / unbind feature through sysfs,
Suppress the bind / unbind calls using driver attribute.
Change-Id: I13a61826435e16849aeb15ebdaa792c849ba45c2
Signed-off-by: Santosh Mardi <gsantosh@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Add support for multiple clients in qoslat by maintaining a
list of devices and aggregating across them during updates.
Change-Id: I3e9970b324938eecdd85f3970ba3acc2c5880fdf
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Move qoslat to late init to ensure all resources it
needs have finished probing.
Change-Id: Ib4f2315aae186f64f531388b2b989c7db9970050
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
The qoslat devfreq device does not implement the
get_dev_status() devfreq profile API. This will
lead to a crash if a governor tries to invoke
devfreq_update_stats(). Implement the API to prevent
this from happening.
Change-Id: I668c3b55a1585d8eb561da03475e1567b194feef
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
lockdep is detecting possible circular locking dependency
due to state_lock mutex as shown below:
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(devfreq_list_lock);
lock(state_lock#2);
lock(devfreq_list_lock);
lock(state_lock#2);
*** DEADLOCK ***
Below is partial call stacks (in reverse order) showing relevant
locking paths:
Call stack for CPU0:
devfreq_bw_hwmon_ev_handler+0x4c/0x5e0 [may acquire &state_lock]
devfreq_add_device+0x418/0x538 [may acquire &devfreq_list_lock]
devfreq_add_icc+0x41c/0x528
devfreq_icc_probe+0x20/0x30
Call stack for CPU1:
devfreq_add_governor+0x3c/0x260 [may acquire &devfreq_list_lock]
register_bw_hwmon+0x1e8/0x248 [may acquire &state_lock]
bimc_bwmon_driver_probe+0x310/0x408
Practically, this race is not possible, since devfreq_add_device first
tries to find the governor, and only if it is succeeds,
devfreq_bw_hwmon_ev_handler is called. And the governor would be found
only if devfreq_add_governor has added it priorly.
We have mechanism(initcall_level) in place to make sure that
governor is added before devfreq_add_device happens.
In attempt to quiet the lockdep warning, below fix is adopted:
Since state_lock mutex has different purpose, introduce a new
event_handle_lock mutex for devfreq_bw_hwmon_ev_handler
to avoid this warning.
Change-Id: I53c4514aa2357bf39e9405683f5e73ada0160ba5
Signed-off-by: Abhishek Shah <abhshah@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
When using the %*pbl printk format a special macro is needed to decode
the cpu mask and format it correctly.
Fixes: 74b58a6 ("PM / devfreq: memlat: Aggregate memlat mons under a controller")
Change-Id: Ic0dedbada86cf9cec02b1a5292aaf05f50c0c42f
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Fix the following warning from modpost:
WARNING: modpost: missing MODULE_LICENSE() in drivers/devfreq/arm-memlat-mon.o
Fixes: a4e0d26 ("PM / devfreq: Introduce a memory-latency governor")
Change-Id: Ic0dedbad2d3462ee407e5f2d5bcb9ded6e6ac83f
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Several devfreq governors print to kernel logs for each
device that registers. Update them to only print when
debugging is enabled.
Change-Id: Id7cf8c9ef70363cdfd4fc1fc72b109f28ab07288
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
A core-dev table is explicity defined in each memlat device
today. However, some devices may want to share the same core-dev
table so add support for parsing a shared core-dev table that
can be used across multiple memlat devices.
Change-Id: I5ee0d439c199a6414d2adaeed8328f3dc99b2a84
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
In register_common(), there's unnecessary code used to find
which device tree node has the core-dev property. Refactor
this to reduce complexity.
Change-Id: Ib3475272b25e898ad23f9f5a4412d90cd889a356
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
This change reverts commit 5e33579a6a9c ("devfreq: Do not allow tunable
updates when device is suspended") and commit db4f4a0933ce ("PM/devfreq:
Do not switch governors from sysfs when device is suspended").
Userspace updates to sysfs nodes like governor and polling_interval are
currently not allowed when there is a non-zero suspend count. This
requirement existed in the past due to hardware dependencies in some
devfreq drivers. However, these hardware dependencies should be handled
by the relevant devfreq drivers rather than the devfreq framework so
remove the suspend count checks in these devfreq framework sysfs APIs.
Change-Id: I8010b519d72784a6b90955acc7899c1a520aa7e7
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Tunables update accesses the monitor's registers for some devices.
When the device is suspended, the clocks for those devices are turned
off and updating the tunables of a suspended device causes the device
to crash. Fix this by checking the flag dev_suspended before allowing
the tunable updates.
Change-Id: I3a0938edf761a9ea6d39ffeec19f35c228d5c306
Signed-off-by: Rama Aparna Mallavarapu <aparnam@codeaurora.org>
[avajid@codeaurora.org: updated to utilize suspend_count from upstream and resolved minor merge conflicts]
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
There is a possibility to switch the governors from sysfs even when the
device is in suspended state. This can cause a NOC error at times when
trying to access the device's monitor registers in a suspended state. This
change fixes this issue. Utilize suspend_count to know if the device is in
suspended state or not. Check if the device is suspended before switching
the governor from sysfs.
Change-Id: I15055aa51daa35272be4667e5bafb8ccd7933098
Signed-off-by: Rama Aparna Mallavarapu <aparnam@codeaurora.org>
[avajid@codeaurora.org: updated to utilize suspend_count from upstream, resolved minor merge conflicts and updated commit text]
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
There is a race condition when the event governor_store is being executed
from sysfs and the device issues a suspend. The devfreq data structures
would become stale when the suspend tries to access them in the middle
of the governor_store operation. Fix this issue by taking a lock around
suspend and resume operations so that these operations are not concurrent
with the other events from sysfs.
Change-Id: Ifa0e93915a920cec3e0429966328a1128d61098b
Signed-off-by: Rama Aparna Mallavarapu <aparnam@codeaurora.org>
[avajid@codeaurora.org: updated to merge better with upstream changes and removed renaming since done in previous commit]
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Currently, concurrent writes to sysfs entries leave the possibility
for race conditions within the devfreq framework. For example,
concurrently executing max_freq_store and governor_store can result
in attempting to perform an update_devfreq() before the new governor's
start handler can be executed.
A more concrete case is a race between polling_interval_store and
governor_store. Because no lock is used after calling into the event
handler of the old governor and there's nothing preventing work from
being queued after the monitor is stopped, it's possible to
accidentally cause delayed work to be queued on the governor being
switched to. This can be seen if you create two threads, one which
changes a device's governor between simple_ondemand and performance,
and one which changes its polling interval between 45 and 50.
All of these races can be addressed with the introduction of a lock
that prevents sysfs operations from interleaving in this fashion.
Change-Id: Ia6887dcb2d69dc2576837a6c09fed55a28943abc
Signed-off-by: Jonathan Avila <avilaj@codeaurora.org>
[avajid@codeaurora.org: renamed to event lock and only used when CONFIG_QCOM_DEVFREQ_ICC is enabled]
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
[ Upstream commit 62453f1ba5d5def9d58e140a50f3f168f028da38 ]
Use the more accurate returned new_freq as resume_freq.
It's the same as how devfreq->previous_freq was updated.
Fixes: 83f8ca45afbf0 ("PM / devfreq: add support for suspend/resume of a devfreq device")
Signed-off-by: Dong Aisheng <aisheng.dong@nxp.com>
Signed-off-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I84195ac392de63fb31647e0eb8a140fa6d742df2
With commit commit 921884ca1498 ("devfreq: Fix PM
callbacks to support zero frequency"), suspend_freq
of zero is considered valid.
However, the devfreq_set_target is called with zero frequency.
To avoid this, do not call devfreq_set_target if the
suspend_freq is less than the minimun allowed frequency.
Change-Id: I506165d28395eb67138ae2dac53e842357809bbb
Signed-off-by: Shreyas K K <shrekk@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
It is perfectly valid for a device to have a zero frequency, usually
it denotes a power collapse. To support this, fix the PM suspend
callback to allow the suspend_freq to be zero.
Change-Id: Id10f95b59b219e82e0695f80361aa6b35cdc5137
Signed-off-by: Shreyas K K <shrekk@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
commit ab8f58ad72 ("PM / devfreq: Set
min/max_freq when adding the devfreq device") introduces a change
where an error in finding the ceil or floor is returned as a
zero frequency. This return of zero is treated as error condition
from the callsites.
It is perfectly valid for a device to have a zero frequency, usually
it denotes a power collapse. This valid return of zero frequency ends
up being treated as an error condition.
Fix it.
Change-Id: I5eb6fa622d6fe09207746b96dce05d0b58b233dc
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shreyas K K <shrekk@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
commit e1e047ace8cef6d143f38c7d769753f133becbe6 upstream.
Commit 2abb0d5268ae ("PM / devfreq: Lock devfreq in trans_stat_show")
revealed a missing locking while calling devfreq_update_status() function
during suspend/resume cycle.
Code analysis revealed that devfreq_set_target() function was called
without needed locks held for setting device specific suspend_freq if such
has been defined. This patch fixes that by adding the needed locking, what
fixes following kernel warning on Exynos4412-based OdroidU3 board during
system suspend:
PM: suspend entry (deep)
Filesystems sync: 0.002 seconds
Freezing user space processes ... (elapsed 0.001 seconds) done.
OOM killer disabled.
Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
------------[ cut here ]------------
WARNING: CPU: 2 PID: 1385 at drivers/devfreq/devfreq.c:204 devfreq_update_status+0xc0/0x188
Modules linked in:
CPU: 2 PID: 1385 Comm: rtcwake Not tainted 5.4.0-rc6-next-20191111 #6848
Hardware name: SAMSUNG EXYNOS (Flattened Device Tree)
[<c0112588>] (unwind_backtrace) from [<c010e070>] (show_stack+0x10/0x14)
[<c010e070>] (show_stack) from [<c0afb010>] (dump_stack+0xb4/0xe0)
[<c0afb010>] (dump_stack) from [<c01272e0>] (__warn+0xf4/0x10c)
[<c01272e0>] (__warn) from [<c01273a8>] (warn_slowpath_fmt+0xb0/0xb8)
[<c01273a8>] (warn_slowpath_fmt) from [<c07d105c>] (devfreq_update_status+0xc0/0x188)
[<c07d105c>] (devfreq_update_status) from [<c07d2d70>] (devfreq_set_target+0xb0/0x15c)
[<c07d2d70>] (devfreq_set_target) from [<c07d3598>] (devfreq_suspend+0x2c/0x64)
[<c07d3598>] (devfreq_suspend) from [<c05de0b0>] (dpm_suspend+0xa4/0x57c)
[<c05de0b0>] (dpm_suspend) from [<c05def74>] (dpm_suspend_start+0x98/0xa0)
[<c05def74>] (dpm_suspend_start) from [<c0195b58>] (suspend_devices_and_enter+0xec/0xc74)
[<c0195b58>] (suspend_devices_and_enter) from [<c0196a20>] (pm_suspend+0x340/0x410)
[<c0196a20>] (pm_suspend) from [<c019480c>] (state_store+0x6c/0xc8)
[<c019480c>] (state_store) from [<c033fc50>] (kernfs_fop_write+0x10c/0x228)
[<c033fc50>] (kernfs_fop_write) from [<c02a6d3c>] (__vfs_write+0x30/0x1d0)
[<c02a6d3c>] (__vfs_write) from [<c02a9afc>] (vfs_write+0xa4/0x180)
[<c02a9afc>] (vfs_write) from [<c02a9d58>] (ksys_write+0x60/0xd8)
[<c02a9d58>] (ksys_write) from [<c0101000>] (ret_fast_syscall+0x0/0x28)
Exception stack(0xed3d7fa8 to 0xed3d7ff0)
...
irq event stamp: 9667
hardirqs last enabled at (9679): [<c0b1e7c4>] _raw_spin_unlock_irq+0x20/0x58
hardirqs last disabled at (9698): [<c0b16a20>] __schedule+0xd8/0x818
softirqs last enabled at (9694): [<c01026fc>] __do_softirq+0x4fc/0x5fc
softirqs last disabled at (9719): [<c012fe68>] irq_exit+0x16c/0x170
---[ end trace 41ac5b57d046bdbc ]---
------------[ cut here ]------------
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Acked-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: Ie7a08b1ea9f1efc5b4f2352968d4f822059ca16a
This patch add basic tracing of the devfreq workqueue and delayed work.
It aims to capture changes of the polling intervals and device state.
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I58e05444446d821d8dbe14cde18da9cbc05e11fb
The patch adds a new file for with trace events for devfreq
framework. They are used for performance analysis of the framework.
It also contains updates in MAINTAINERS file adding new entry for
devfreq maintainers.
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: Id252a3809cd0d210e3af027141b3ff7572fbfcc7
If the new governor fails to start, switch back to old governor so that the
devfreq state is not left in some weird limbo.
[Myungjoo: assume fatal on revert failure and set df->governor to NULL]
Signed-off-by: Sibi Sankar <sibis@codeaurora.org>
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: If9c35dca5d07fbfff6de236e7b2bfb5fd299abc7
Following up with complaints on inconsistent indentation from
Yangtao Li, this fixes indentation inconsistency.
In principle, this tries to put arguments aligned to the left
including the first argument except for the case where
the first argument is on the far-right side.
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Acked-by: Yangtao Li <tiny.windzz@gmail.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: Ib337712266c82f8bf70c3cbd0ed61f40d300db73
devm_kzalloc() could fail, so insert a check of its return value. And
if it fails, returns -ENOMEM.
Signed-off-by: Yangtao Li <tiny.windzz@gmail.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I8d305a8929b41854b4145dc9e04fa01e31b2af20
'devfreq' is malloced in devfreq_add_device() and should be freed in
the error handling cases, otherwise it will cause memory leak.
Signed-off-by: Yangtao Li <tiny.windzz@gmail.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I2ce1c43a550abb1e31eb3024222a3be7e8d67fe7
This patch adds implementation for global suspend/resume for
devfreq framework. System suspend will next use these functions.
Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I58de77b6d59f2508e254ccbeec49c0be82dd3c8e
The patch prepares devfreq device for handling suspend/resume
functionality. The new fields will store needed information during this
process. Devfreq framework handles opp-suspend DT entry and there is no
need of modyfications in the drivers code. It uses atomic variables to
make sure no race condition affects the process.
Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I853e345c565593d7af3a8efc0c7c60514f9007f3
The refactoring is needed for the new client in devfreq: suspend.
To avoid code duplication, move it to the new local function
devfreq_set_target.
Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I735f1f380a84d5151a6b22b4824c307db28f2a82
kfree has taken the null pointer into account. hence it is safe
to remove the redundant null pointer check before kfree.
Signed-off-by: zhong jiang <zhongjiang@huawei.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I5d8568549fe93f131c390742fcdfcf3d290c65f4
Drop the custom MIN/MAX macros in favour of the standard min/max from
kernel.h
Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Change-Id: I60ef1488ea87b47ef05c817052dc86b4695c6ffd
Will aid debugging lockups in perf_event_read_value()
track the cpu being ipi'd.
Change-Id: Ia948f31bb2d91bca6144c0c50f8f66bd9c1459fe
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Signed-off-by: UtsavBalar1231 <utsavbalar1231@gmail.com>
Phandle lookups still occur frequently after boot (like in the regulator
subsystem), and they can be quite expensive if the device tree is
complex. Lookups disable IRQs and have been observed to take over a
millisecond on a mobile arm64 device, which is very bad for system
latency.
Keep the phandle cache around after boot to retain O(1) lookup times.
Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
Currently, thread flag tests are unordered with respect to flag changes,
which results in thread flag changes not becoming immediately visible to
other CPUs. On a weakly-ordered CPU, this is most noticeable with the
TIF_NEED_RESCHED flag and optimistic lock spinners, where the preemptoff
tracer shows an optimistic lock spinner will often elapse its scheduling
quantum despite checking TIF_NEED_RESCHED on every loop iteration. This
leads to scheduling delays and latency spikes, especially when disabling
preemption is involved, as is the case for optimistic lock spinning.
Making the thread flag helpers ordered with respect to test operations
resolves the issue seen in the preemptoff tracer. Now, optimistic lock
spinners bail out in a timely manner, and other TIF_NEED_RESCHED users
will benefit similarly.
Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
There is noticeable scheduling latency and heavy zone lock contention
stemming from rmqueue_bulk's single hold of the zone lock while doing
its work, as seen with the preemptoff tracer. There's no actual need for
rmqueue_bulk() to hold the zone lock the entire time; it only does so
for supposed efficiency. As such, we can relax the zone lock and even
reschedule when IRQs are enabled in order to keep the scheduling delays
and zone lock contention at bay. Forward progress is still guaranteed,
as the zone lock can only be relaxed after page removal.
With this change, rmqueue_bulk() no longer appears as a serious offender
in the preemptoff tracer, and system latency is noticeably improved.
Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
The page allocator processes free pages in groups of pageblocks, where
the size of a pageblock is typically quite large (1024 pages without
hugetlbpage support). Pageblocks are processed atomically with the zone
lock held, which can cause severe scheduling delays on both the CPU
going through the pageblock and any other CPUs waiting to acquire the
zone lock. A frequent offender is move_freepages_block(), which is used
by rmqueue() for page allocation.
As it turns out, there's no requirement for pageblocks to be so large,
so the pageblock order can simply be reduced to ease the scheduling
delays and zone lock contention. PAGE_ALLOC_COSTLY_ORDER is used as a
reasonable setting to ensure non-costly page allocation requests can
still be serviced without always needing to free up more than one
pageblock's worth of pages at a time.
This has a noticeable effect on overall system latency when memory
pressure is elevated. The various mm functions which operate on
pageblocks no longer appear in the preemptoff tracer, where previously
they would spend up to 100 ms on a mobile arm64 CPU processing a
pageblock with preemption disabled and the zone lock held.
Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
