summaryrefslogtreecommitdiffstats
path: root/tools/tracing/rtla (follow)
Commit message (Collapse)AuthorAgeFilesLines
* rtla/timerlat: Fix histogram ALL for zero samplesTomas Glozar2024-12-271-81/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | rtla timerlat hist currently computers the minimum, maximum and average latency even in cases when there are zero samples. This leads to nonsensical values being calculated for maximum and minimum, and to divide by zero for average. A similar bug is fixed by 01b05fc0e5f3 ("rtla/timerlat: Fix histogram report when a cpu count is 0") but the bug still remains for printing the sum over all CPUs in timerlat_print_stats_all. The issue can be reproduced with this command: $ rtla timerlat hist -U -d 1s Index over: count: min: avg: max: Floating point exception (core dumped) (There are always no samples with -U unless the user workload is created.) Fix the bug by omitting max/min/avg when sample count is zero, displaying a dash instead, just like we already do for the individual CPUs. The logic is moved into a new function called format_summary_value, which is used for both the individual CPUs and for the overall summary. Cc: stable@vger.kernel.org Link: https://lore.kernel.org/20241127134130.51171-1-tglozar@redhat.com Fixes: 1462501c7a8 ("rtla/timerlat: Add a summary for hist mode") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* Merge tag 'trace-tools-v6.13' of ↵Linus Torvalds2024-11-229-66/+327
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracing tools updates from Steven Rostedt: - Add ':' to getopt option 'trace-buffer-size' in timerlat_hist for consistency - Remove unused sched_getattr define - Rename sched_setattr() helper to syscall_sched_setattr() to avoid conflicts - Update counters to long from int to avoid overflow - Add libcpupower dependency detection - Add --deepest-idle-state to timerlat to limit deep idle sleeps - Other minor clean ups and documentation changes * tag 'trace-tools-v6.13' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: verification/dot2: Improve dot parser robustness tools/rtla: Improve exception handling in timerlat_load.py tools/rtla: Enhance argument parsing in timerlat_load.py tools/rtla: Improve code readability in timerlat_load.py rtla/timerlat: Do not set params->user_workload with -U rtla: Documentation: Mention --deepest-idle-state rtla/timerlat: Add --deepest-idle-state for hist rtla/timerlat: Add --deepest-idle-state for top rtla/utils: Add idle state disabling via libcpupower rtla: Add optional dependency on libcpupower tools/build: Add libcpupower dependency detection rtla/timerlat: Make timerlat_hist_cpu->*_count unsigned long long rtla/timerlat: Make timerlat_top_cpu->*_count unsigned long long tools/rtla: fix collision with glibc sched_attr/sched_set_attr tools/rtla: drop __NR_sched_getattr rtla: Fix consistency in getopt_long for timerlat_hist rv: Fix a typo tools/rv: Correct the grammatical errors in the comments tools/rv: Correct the grammatical errors in the comments rtla: use the definition for stdout fd when calling isatty()
| * tools/rtla: Improve exception handling in timerlat_load.pyfurkanonder2024-11-191-16/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The enhancements made to timerlat_load.py are intended to improve the script's exception handling. Summary of the changes: - Specific exceptions are now caught for CPU affinity and priority settings, with clearer error messages provided. - The timerlat file descriptor opening now includes handling for PermissionError and OSError, with informative messages. - In the infinite loop, generic exceptions have been replaced with specific types like KeyboardInterrupt and IOError, improving feedback. Before: $ sudo python timerlat_load.py 122 Error setting affinity After: $ sudo python timerlat_load.py 122 Error setting affinity: [Errno 22] Invalid argument Before: $ sudo python timerlat_load.py 1 -p 950 Error setting priority After: $ sudo python timerlat_load.py 1 -p 950 Error setting priority: [Errno 22] Invalid argument Before: $ python timerlat_load.py 1 Error opening timerlat fd, did you run timerlat -U? After: $ python timerlat_load.py 1 Permission denied. Please check your access rights. Cc: "lgoncalv@redhat.com" <lgoncalv@redhat.com> Cc: "jkacur@redhat.com" <jkacur@redhat.com> Link: https://lore.kernel.org/Q_k1s4hBtUy2px8ou0QKenjEK2_T_LoV8IxAE79aBakBogb-7uHp2fpET3oWtI1t3dy8uKjWeRzQOdKNzIzOOpyM4OjutJOriZ9TrGY6b-g=@protonmail.com Signed-off-by: Furkan Onder <furkanonder@protonmail.com> Reviewed-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * tools/rtla: Enhance argument parsing in timerlat_load.pyfurkanonder2024-11-191-5/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The enhancements made to timerlat_load.py are aimed at improving the clarity of argument parsing. Summary of Changes: - The cpu argument is now specified as an integer type in the argument parser to enforce input validation, and the construction of affinity_mask has been simplified to directly use the integer value of args.cpu. - The prio argument is similarly updated to be of integer type for consistency and validation, eliminating the need for the conversion of args.prio to an integer, as this is now handled by the argument parser. Cc: "jkacur@redhat.com" <jkacur@redhat.com> Cc: "lgoncalv@redhat.com" <lgoncalv@redhat.com> Link: https://lore.kernel.org/QfgO7ayKD9dsLk8_ZDebkAV0OF7wla7UmasbP9CBmui_sChOeizy512t3RqCHTjvQoUBUDP8dwEOVCdHQ5KvVNEiP69CynMY94SFDERWl94=@protonmail.com Signed-off-by: Furkan Onder <furkanonder@protonmail.com> Reviewed-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * tools/rtla: Improve code readability in timerlat_load.pyfurkanonder2024-11-191-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The enhancements made to timerlat_load.py are intended to improve the script's robustness and readability. Summary of the changes: - Unnecessary semicolons at the end of lines have been removed. - Parentheses surrounding the if statement checking args.prio have been eliminated. - String concatenation for constructing timerlat_path has been replaced with an f-string. - Spacing in a multiplication expression has been adjusted for improved clarity. Cc: "jkacur@redhat.com" <jkacur@redhat.com> Cc: "lgoncalv@redhat.com" <lgoncalv@redhat.com> Link: https://lore.kernel.org/j2B-ted7pv3TaldTyqfIHrMmjq2fVyBFgnu3TskiQJsyRzy9loPTVVJoqHnrCWu5T88MDIFc612jUglH6Sxkdg9LN-I1XuITmoL70uECmus=@protonmail.com Signed-off-by: Furkan Onder <furkanonder@protonmail.com> Reviewed-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla/timerlat: Do not set params->user_workload with -UTomas Glozar2024-11-192-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since commit fb9e90a67ee9 ("rtla/timerlat: Make user-space threads the default"), rtla-timerlat has been defaulting to params->user_workload if neither that or params->kernel_workload is set. This has unintentionally made -U, which sets only params->user_hist/top but not params->user_workload, to behave like -u unless -k is set, preventing the user from running a custom workload. Example: $ rtla timerlat hist -U -c 0 & [1] 7413 $ python sample/timerlat_load.py 0 Error opening timerlat fd, did you run timerlat -U? $ ps | grep timerlatu 7415 pts/4 00:00:00 timerlatu/0 Fix the issue by checking for params->user_top/hist instead of params->user_workload when setting default thread mode. Link: https://lore.kernel.org/20241021123140.14652-1-tglozar@redhat.com Fixes: fb9e90a67ee9 ("rtla/timerlat: Make user-space threads the default") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla: Documentation: Mention --deepest-idle-stateTomas Glozar2024-10-171-0/+4
| | | | | | | | | | | | | | | | | | Add --deepest-idle-state to manpage and mention libcpupower dependency in README.txt. Link: https://lore.kernel.org/20241017140914.3200454-7-tglozar@redhat.com Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla/timerlat: Add --deepest-idle-state for histTomas Glozar2024-10-171-1/+41
| | | | | | | | | | | | | | | | Support limiting deepest idle state also for timerlat-hist. Link: https://lore.kernel.org/20241017140914.3200454-6-tglozar@redhat.com Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla/timerlat: Add --deepest-idle-state for topTomas Glozar2024-10-171-1/+41
| | | | | | | | | | | | | | | | | | Add option to limit deepest idle state on CPUs where timerlat is running for the duration of the workload. Link: https://lore.kernel.org/20241017140914.3200454-5-tglozar@redhat.com Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla/utils: Add idle state disabling via libcpupowerTomas Glozar2024-10-172-0/+163
| | | | | | | | | | | | | | | | | | | | Add functions to utils.c to disable idle states through functions of libcpupower. This will serve as the basis for disabling idle states per cpu when running timerlat. Link: https://lore.kernel.org/20241017140914.3200454-4-tglozar@redhat.com Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla: Add optional dependency on libcpupowerTomas Glozar2024-10-172-0/+12
| | | | | | | | | | | | | | | | | | If libcpupower is present, set HAVE_LIBCPUPOWER_SUPPORT macro to allow features depending on libcpupower in rtla. Link: https://lore.kernel.org/20241017140914.3200454-3-tglozar@redhat.com Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla/timerlat: Make timerlat_hist_cpu->*_count unsigned long longTomas Glozar2024-10-111-9/+9
| | | | | | | | | | | | | | | | | | Do the same fix as in previous commit also for timerlat-hist. Link: https://lore.kernel.org/20241011121015.2868751-2-tglozar@redhat.com Reported-by: Attila Fazekas <afazekas@redhat.com> Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla/timerlat: Make timerlat_top_cpu->*_count unsigned long longTomas Glozar2024-10-111-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most fields of struct timerlat_top_cpu are unsigned long long, but the fields {irq,thread,user}_count are int (32-bit signed). This leads to overflow when tracing on a large number of CPUs for a long enough time: $ rtla timerlat top -a20 -c 1-127 -d 12h ... 0 12:00:00 | IRQ Timer Latency (us) | Thread Timer Latency (us) CPU COUNT | cur min avg max | cur min avg max 1 #43200096 | 0 0 1 2 | 3 2 6 12 ... 127 #43200096 | 0 0 1 2 | 3 2 5 11 ALL #119144 e4 | 0 5 4 | 2 28 16 The average latency should be 0-1 for IRQ and 5-6 for thread, but is reported as 5 and 28, about 4 to 5 times more, due to the count overflowing when summed over all CPUs: 43200096 * 127 = 5486412192, however, 1191444898 (= 5486412192 mod MAX_INT) is reported instead, as seen on the last line of the output, and the averages are thus ~4.6 times higher than they should be (5486412192 / 1191444898 = ~4.6). Fix the issue by changing {irq,thread,user}_count fields to unsigned long long, similarly to other fields in struct timerlat_top_cpu and to the count variable in timerlat_top_print_sum. Link: https://lore.kernel.org/20241011121015.2868751-1-tglozar@redhat.com Reported-by: Attila Fazekas <afazekas@redhat.com> Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * tools/rtla: fix collision with glibc sched_attr/sched_set_attrJan Stancek2024-10-112-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glibc commit 21571ca0d703 ("Linux: Add the sched_setattr and sched_getattr functions") now also provides 'struct sched_attr' and sched_setattr() which collide with the ones from rtla. In file included from src/trace.c:11: src/utils.h:49:8: error: redefinition of ‘struct sched_attr’ 49 | struct sched_attr { | ^~~~~~~~~~ In file included from /usr/include/bits/sched.h:60, from /usr/include/sched.h:43, from /usr/include/tracefs/tracefs.h:10, from src/trace.c:4: /usr/include/linux/sched/types.h:98:8: note: originally defined here 98 | struct sched_attr { | ^~~~~~~~~~ Define 'struct sched_attr' conditionally, similar to what strace did: https://lore.kernel.org/all/20240930222913.3981407-1-raj.khem@gmail.com/ and rename rtla's version of sched_setattr() to avoid collision. Link: https://lore.kernel.org/8088f66a7a57c1b209cd8ae0ae7c336a7f8c930d.1728572865.git.jstancek@redhat.com Signed-off-by: Jan Stancek <jstancek@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * tools/rtla: drop __NR_sched_getattrJan Stancek2024-10-111-18/+14
| | | | | | | | | | | | | | | | | | | | It's not used since commit 084ce16df0f0 ("tools/rtla: Remove unused sched_getattr() function"). Link: https://lore.kernel.org/c355dc9ad23470098d6a8d0f31fbd702551c9ea8.1728552769.git.jstancek@redhat.com Signed-off-by: Jan Stancek <jstancek@redhat.com> Reviewed-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla: Fix consistency in getopt_long for timerlat_histGabriele Monaco2024-10-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit e9a4062e1527 ("rtla: Add --trace-buffer-size option") adds a new long option to rtla utilities, but among all affected files, timerlat_hist misses a trailing `:` in the corresponding short option inside the getopt string (e.g. `\3:`). This patch propagates the `:`. Although this change is not functionally required, it improves consistency and slightly reduces the likelihood a future change would introduce a problem. Cc: John Kacur <jkacur@redhat.com> Cc: Luis Goncalves <lgoncalv@redhat.com> Cc: Tomas Glozar <tglozar@redhat.com> Link: https://lore.kernel.org/20240926143417.54039-1-gmonaco@redhat.com Signed-off-by: Gabriele Monaco <gmonaco@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
| * rtla: use the definition for stdout fd when calling isatty()Eder Zulian2024-10-042-2/+2
| | | | | | | | | | | | | | | | | | Use the STDOUT_FILENO definition when testing whether the standard output file descriptor refers to a terminal (for better redability). Link: https://lore.kernel.org/20240813142338.376039-1-ezulian@redhat.com Signed-off-by: Eder Zulian <ezulian@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla: Fix the help text in osnoise and timerlat top toolsEder Zulian2024-10-032-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The help text in osnoise top and timerlat top had some minor errors and omissions. The -d option was missing the 's' (second) abbreviation and the error message for '-d' used '-D'. Cc: stable@vger.kernel.org Fixes: 1eceb2fc2ca54 ("rtla/osnoise: Add osnoise top mode") Fixes: a828cd18bc4ad ("rtla: Add timerlat tool and timelart top mode") Link: https://lore.kernel.org/20240813155831.384446-1-ezulian@redhat.com Suggested-by: Tomas Glozar <tglozar@redhat.com> Reviewed-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Eder Zulian <ezulian@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | tools/rtla: Fix installation from out-of-tree buildBen Hutchings2024-10-031-1/+1
|/ | | | | | | | | | | | | rtla now supports out-of-tree builds, but installation fails as it still tries to install the rtla binary from the source tree. Use the existing macro $(RTLA) to refer to the binary. Link: https://lore.kernel.org/ZudubuoU_JHjPZ7w@decadent.org.uk Fixes: 01474dc706ca ("tools/rtla: Use tools/build makefiles to build rtla") Reviewed-by: Tomas Glozar <tglozar@redhat.com> Tested-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Ben Hutchings <benh@debian.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/osnoise: Prevent NULL dereference in error handlingDan Carpenter2024-08-091-7/+4
| | | | | | | | | | | | | | If the "tool->data" allocation fails then there is no need to call osnoise_free_top() and, in fact, doing so will lead to a NULL dereference. Cc: stable@vger.kernel.org Cc: John Kacur <jkacur@redhat.com> Cc: "Luis Claudio R. Goncalves" <lgoncalv@redhat.com> Cc: Clark Williams <williams@redhat.com> Fixes: 1eceb2fc2ca5 ("rtla/osnoise: Add osnoise top mode") Link: https://lore.kernel.org/f964ed1f-64d2-4fde-ad3e-708331f8f358@stanley.mountain Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* Merge tag 'perf-tools-fixes-for-v6.11-2024-07-23' of ↵Linus Torvalds2024-07-241-1/+2
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools Pull perf tools fixes from Namhyung Kim: "Two fixes for building perf and other tools: - Fix breakage in tracing tools due to pkg-config for libtrace{event,fs} - Fix build of perf when libunwind is used" * tag 'perf-tools-fixes-for-v6.11-2024-07-23' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools: perf dso: Fix build when libunwind is enabled tools/latency: Use pkg-config in lib_setup of Makefile.config tools/rtla: Use pkg-config in lib_setup of Makefile.config tools/verification: Use pkg-config in lib_setup of Makefile.config tools: Make pkg-config dependency checks usable by other tools perf build: Warn if libtracefs is not found
| * tools/rtla: Use pkg-config in lib_setup of Makefile.configGuilherme Amadio2024-07-171-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | This allows to build against libtraceevent and libtracefs installed in non-standard locations. Signed-off-by: Guilherme Amadio <amadio@gentoo.org> Tested-by: Thorsten Leemhuis <linux@leemhuis.info> Tested-by: Leo Yan <leo.yan@arm.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Link: https://lore.kernel.org/r/20240712194511.3973899-5-amadio@gentoo.org Signed-off-by: Namhyung Kim <namhyung@kernel.org>
* | rtla/osnoise: Better report when histogram is emptyLuis Claudio R. Goncalves2024-06-211-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When osnoise hist does not observe any samples above the threshold, no entries are recorded and the final report shows empty entries for the usual statistics (count, min, max, avg): [~]# osnoise hist -d 5s -T 500 # RTLA osnoise histogram # Time unit is microseconds (us) # Duration: 0 00:00:05 Index over: count: min: avg: max: That could lead users to confusing interpretations of the results. A simple solution is to report 0 for count and the statistics, making it clear that no noise (above the defined threshold) was observed: [~]# osnoise hist -d 5s -T 500 # RTLA osnoise histogram # Time unit is microseconds (us) # Duration: 0 00:00:05 Index over: 0 count: 0 min: 0 avg: 0 max: 0 Link: https://lkml.kernel.org/r/Zml6JmH5cbS7-HfZ@uudg.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: John Kacur <jkacur@redhat.com> Cc: Clark Williams <williams@redhat.com> Reviewed-by: John Kacur <jkacur@redhat.com> Signed-off-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* | rtla/osnoise: Use pretty formatting only on interactive ttyLuis Claudio R. Goncalves2024-06-211-4/+15
|/ | | | | | | | | | | | | | | | | osnoise top performs background/font color formatting that could make the text output confusing if not on a terminal. Use the changes from commit f5c0cdad6684a ("rtla/timerlat: Use pretty formatting only on interactive tty") as an inspiration to fix this problem. Apply the formatting only if running on a tty, and not in quiet mode. Link: https://lkml.kernel.org/r/Zmb-yP_3EDHliI8Z@uudg.org Suggested-by: Daniel Bristot de Oliveira <bristot@kernel.org> Reviewed-by: John Kacur <jkacur@redhat.com> Reviewed-by: Clark Williams <williams@redhat.com> Signed-off-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla: Fix -t\--trace[=file]John Kacur2024-05-164-20/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The -t option has an optional argument. The usual case is for a short option to be specified without an '=' and for the long version to be specified with an '=' Various forms of this do not work as expected. For example: rtla timerlat hist -T50 -tfile.txt will result in a truncated file name of "ile.txt" Another example is that the long form without the '=' will result in the default file name instead of the requested file name. This patch properly parses the optional argument with and without '=' and with and without spaces for the short form. This patch was also tested using -t and --trace without providing a file name both as the last requested option and with a following long and short option. For example: rtla timerlat hist -T50 -t -u rtla timerlat hist -T50 --trace -u This fix is applied to both timerlat top and hist and to osnoise top and hist. Here is the full testing for rtla timerlat hist. Before applying the patch rtla timerlat hist -T50 -t=file.txt Works as expected, "file.txt" rtla timerlat hist -T50 -tfile.txt Truncated file name "ile.txt" rtla timerlat hist -T50 -t file.txt Default file name instead of file.txt rtla timerlat hist -T50 --trace=file.txt Truncated file name "ile.txt" rtla timerlat hist -T50 --trace file.txt Default file name "timerlat_trace.txt" instead of "file.txt" After applying the patch: rtla timerlat hist -T50 -t=file.txt Works as expected, "file.txt" rtla timerlat hist -T50 -tfile.txt Works as expected, "file.txt" rtla timerlat hist -T50 -t file.txt Works as expected, "file.txt" rtla timerlat hist -T50 --trace=file.txt Works as expected, "file.txt" rtla timerlat hist -T50 --trace file.txt Works as expected, "file.txt" In addition the following tests were performed to make sure that the default file name worked as expected including with trailing options. rtla timerlat hist -T50 -t Works as expected "timerlat_trace.txt" rtla timerlat hist -T50 --trace Works as expected "timerlat_trace.txt" rtla timerlat hist -T50 -t -u Works as expected "timerlat_trace.txt" rtla timerlat hist -T50 --trace -u Works as expected "timerlat_trace.txt" Link: https://lkml.kernel.org/r/20240515183024.59985-1-jkacur@redhat.com Cc: Daniel Bristot de Oliveria <bristot@kernel.org> Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat: Fix histogram report when a cpu count is 0John Kacur2024-05-161-18/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On short runs it is possible to get no samples on a cpu, like this: # rtla timerlat hist -u -T50 Index IRQ-001 Thr-001 Usr-001 IRQ-002 Thr-002 Usr-002 2 1 0 0 0 0 0 33 0 1 0 0 0 0 36 0 0 1 0 0 0 49 0 0 0 1 0 0 52 0 0 0 0 1 0 over: 0 0 0 0 0 0 count: 1 1 1 1 1 0 min: 2 33 36 49 52 18446744073709551615 avg: 2 33 36 49 52 - max: 2 33 36 49 52 0 rtla timerlat hit stop tracing IRQ handler delay: (exit from idle) 48.21 us (91.09 %) IRQ latency: 49.11 us Timerlat IRQ duration: 2.17 us (4.09 %) Blocking thread: 1.01 us (1.90 %) swapper/2:0 1.01 us ------------------------------------------------------------------------ Thread latency: 52.93 us (100%) Max timerlat IRQ latency from idle: 49.11 us in cpu 2 Note, the value 18446744073709551615 is the same as ~0. Fix this by reporting no results for the min, avg and max if the count is 0. Link: https://lkml.kernel.org/r/20240510190318.44295-1-jkacur@redhat.com Cc: stable@vger.kernel.org Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Suggested-by: Daniel Bristot de Oliveria <bristot@kernel.org> Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla: Add --trace-buffer-size optionDaniel Bristot de Oliveira2024-05-167-5/+68
| | | | | | | | | | | | | | | | | | | Add the option allow the users to set a different buffer size for the trace. For example, in large systems, the user might be interested on reducing the trace buffer to avoid large tracing files. The buffer size is specified in kB, and it is only affecting the tracing instance. The function trace_set_buffer_size() appears on libtracefs v1.6, so increase the minimum required version on Makefile.config. Link: https://lkml.kernel.org/r/e7c9ca5b3865f28e131a49ec3b984fadf2d056c6.1715860611.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat: Make user-space threads the defaultDaniel Bristot de Oliveira2024-05-152-6/+56
| | | | | | | | | | | | | | | | | After ther -u addition, most of the known users are setting it. And it makes sense, as it adds more information, and inherits the default setup for the threads - e.g., cgroups configs. Thus, if the user-space interface is available, enable -u. Otherwise, use the in-kernel thread. Add the -k option to allow the user to request kernel-threads. Link: https://lkml.kernel.org/r/9241d3089de4091b124f780ed832a0e6646cadaa.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla: Add the --warm-up optionDaniel Bristot de Oliveira2024-05-154-40/+115
| | | | | | | | | | | | | | | | On many cases, the results right after the startup are different from the rest of the execution, biasing the results. For example, on osnoise, the scheduler might take some time to adapt to the new busy-loop workload. Add the --warm-up <seconds> option, adding a warm-up phase (in seconds) where the workload is set, but the results are discarded. Link: https://lkml.kernel.org/r/e682d5ce5af90f123bd13220f63d5c3d118a92be.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat: Add a summary for hist modeDaniel Bristot de Oliveira2024-05-151-1/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Like on rtla timerlat top, add an overall summary at the bottom of timerlat hist. For instance: # timerlat hist -c 0-1 -d 10s -E 20 # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:00:10 Index IRQ-000 Thr-000 IRQ-001 Thr-001 6 1 0 0 0 7 1 0 0 0 8 1 0 1 0 9 7 0 0 0 10 16 0 0 0 11 1 0 3 0 15 0 0 3 0 16 0 0 12 0 17 0 0 28 0 18 0 2 26 0 19 1 1 80 1 over: 9973 9998 9848 10000 count: 10001 10001 10001 10001 min: 6 18 8 19 avg: 185 204 95 113 max: 428 450 341 371 ALL: IRQ Thr count: 20002 20002 min: 6 18 avg: 140 159 max: 428 450 Link: https://lkml.kernel.org/r/a6bc06c798f72127edc57d1f99da8d57e1187cee.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Suggested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat: Add a summary for top modeDaniel Bristot de Oliveira2024-05-151-0/+108
| | | | | | | | | | | | | | | | | | | | | | | | While the per-cpu values are the results to take into consideration, the overall system values are also useful. Add a summary at the bottom of rtla timerlat top showing the overall results. For instance: Timer Latency 0 00:00:10 | IRQ Timer Latency (us) | Thread Timer Latency (us) CPU COUNT | cur min avg max | cur min avg max 0 #10003 | 113 19 150 441 | 134 35 170 459 1 #10003 | 63 8 99 462 | 84 15 119 481 2 #10003 | 3 2 89 396 | 21 8 108 414 3 #10002 | 206 11 210 394 | 223 21 228 415 ---------------|----------------------------------------|--------------------------------------- ALL #40011 e0 | 2 137 462 | 8 156 481 Link: https://lkml.kernel.org/r/5eb510d6faeb4ce745e09395196752df75a2dd1a.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Suggested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat: Use pretty formatting only on interactive ttyDaniel Bristot de Oliveira2024-05-151-7/+18
| | | | | | | | | | | | | | | | | | | | | | timerlat top does some background/font color formatting. While useful on terminal, it breaks the output on other formats. For example, when piping the output for pastebin tools, the format strings are printed as characters. For instance: [2;37;40m Timer Latency [0;0;0m 0 00:00:01 | IRQ Timer Latency (us) | Thread Timer Latency (us) [2;30;47mCPU COUNT | cur min avg max | cur min avg max[0;0;0m 0 #1013 | 1 0 1 54 | 5 2 4 57 1 #1013 | 3 0 1 10 | 6 2 4 15 To avoid this problem, do the formatting only if running on a tty, and in !quiet mode. Link: https://lkml.kernel.org/r/8288e1544ceab21557d5dda93a0f00339497c649.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/auto-analysis: Replace \t with spacesDaniel Bristot de Oliveira2024-05-151-46/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When copying timerlat auto-analysis from a terminal to some web pages or chats, the \t are being replaced with a single ' ' or ' ', breaking the output. For example: ## CPU 3 hit stop tracing, analyzing it ## IRQ handler delay: 1.30 us (0.11 %) IRQ latency: 1.90 us Timerlat IRQ duration: 3.00 us (0.24 %) Blocking thread: 1223.16 us (99.00 %) insync:4048 1223.16 us IRQ interference 4.93 us (0.40 %) local_timer:236 4.93 us ------------------------------------------------------------------------ Thread latency: 1235.47 us (100%) Replace \t with spaces to avoid this problem. Link: https://lkml.kernel.org/r/ec7ed2b2809c22ab0dfc8eb7c805ab9cddc4254a.1713968967.git.bristot@kernel.org Cc: stable@vger.kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Fixes: 27e348b221f6 ("rtla/timerlat: Add auto-analysis core") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat: Simplify "no value" printing on topDaniel Bristot de Oliveira2024-05-151-12/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of printing three times the same output, print it only once, reducing lines and being sure that all no values have the same length. It also fixes an extra '\n' when running the with kernel threads, like here: =============== %< ============== Timer Latency 0 00:00:01 | IRQ Timer Latency (us) | Thread Timer Latency (us) CPU COUNT | cur min avg max | cur min avg max 2 #0 | - - - - | 161 161 161 161 3 #0 | - - - - | 161 161 161 161 8 #1 | 54 54 54 54 | - - - -'\n' ---------------|----------------------------------------|--------------------------------------- ALL #1 e0 | 54 54 54 | 161 161 161 =============== %< ============== This '\n' should have been removed with the user-space support that added another '\n' if not running with kernel threads. Link: https://lkml.kernel.org/r/0a4d8085e7cd706733a5dc10a81ca38b82bd4992.1713968967.git.bristot@kernel.org Cc: stable@vger.kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Fixes: cdca4f4e5e8e ("rtla/timerlat_top: Add timerlat user-space support") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* tools/rtla: Add -U/--user-load option to timerlatDaniel Bristot de Oliveira2024-03-203-9/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The timerlat tracer provides an interface for any application to wait for the timerlat's periodic wakeup. Currently, rtla timerlat uses it to dispatch its user-space workload (-u option). But as the tracer interface is generic, rtla timerlat can also be used to monitor any workload that uses it. For example, a user might place their own workload to wait on the tracer interface, and monitor the results with rtla timerlat. Add the -U option to rtla timerlat top and hist. With this option, rtla timerlat will not dispatch its workload but only setting up the system, waiting for a user to dispatch its workload. The sample code in this patch is an example of python application that loops in the timerlat tracer fd. To use it, dispatch: # rtla timerlat -U In a terminal, then run the python program on another terminal, specifying the CPU to run it. For example, setting on CPU 1: #./timerlat_load.py 1 Then rtla timerlat will start printing the statistics of the ./timerlat_load.py app. An interesting point is that the "Ret user Timer Latency" value is the overall response time of the load. The sample load does a memory copy to exemplify that. The stop tracing options on rtla timerlat works in this setup as well, including auto analysis. Link: https://lkml.kernel.org/r/36e6bcf18fe15c7601048fd4c65aeb193c502cc8.1707229706.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Masami Hiramatsu <mhiramat@kernel.org> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* tools/rtla: Use tools/build makefiles to build rtlaDaniel Bristot de Oliveira2024-03-207-145/+244
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use tools/build/ makefiles to build rtla, inheriting the benefits of it. For example, having a proper way to handle dependencies. rtla is built using perf infra-structure when building inside the kernel tree. At this point, rtla diverges from perf in two points: Documentation and tarball generation/build. At the documentation level, rtla is one step ahead, placing the documentation at Documentation/tools/rtla/, using the same build tools as kernel documentation. The idea is to move perf documentation to the same scheme and then share the same makefiles. rtla has a tarball target that the (old) RHEL8 uses. The tarball was kept using a simple standalone makefile for compatibility. The standalone makefile shares most of the code, e.g., flags, with regular buildings. The tarball method was set as deprecated. If necessary, we can make a rtla tarball like perf, which includes the entire tools/build. But this would also require changes in the user side (the directory structure changes, and probably the deps to build the package). Inspired on perf and objtool. Link: https://lkml.kernel.org/r/57563abf2715d22515c0c54a87cff3849eca5d52.1710519524.git.bristot@kernel.org Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Josh Poimboeuf <jpoimboe@kernel.org> Cc: John Kacur <jkacur@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Jiri Olsa <jolsa@kernel.org> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* tools/rtla: Exit with EXIT_SUCCESS when help is invokedJohn Kacur2024-02-124-4/+20
| | | | | | | | | | | | | | | | Fix rtla so that the following commands exit with 0 when help is invoked rtla osnoise top -h rtla osnoise hist -h rtla timerlat top -h rtla timerlat hist -h Link: https://lore.kernel.org/linux-trace-devel/20240203001607.69703-1-jkacur@redhat.com Cc: stable@vger.kernel.org Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* tools/rtla: Replace setting prio with nice for SCHED_OTHERlimingming32024-02-122-3/+5
| | | | | | | | | | | | | Since the sched_priority for SCHED_OTHER is always 0, it makes no sence to set it. Setting nice for SCHED_OTHER seems more meaningful. Link: https://lkml.kernel.org/r/20240207065142.1753909-1-limingming3@lixiang.com Cc: stable@vger.kernel.org Fixes: b1696371d865 ("rtla: Helper functions for rtla") Signed-off-by: limingming3 <limingming3@lixiang.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* tools/rtla: Remove unused sched_getattr() functionDaniel Bristot de Oliveira2024-02-121-6/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | Clang is reporting: $ make HOSTCC=clang CC=clang LLVM_IAS=1 [...] clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS $(pkg-config --cflags libtracefs) -c -o src/utils.o src/utils.c src/utils.c:241:19: warning: unused function 'sched_getattr' [-Wunused-function] 241 | static inline int sched_getattr(pid_t pid, struct sched_attr *attr, | ^~~~~~~~~~~~~ 1 warning generated. Which is correct, so remove the unused function. Link: https://lkml.kernel.org/r/eaed7ba122c4ae88ce71277c824ef41cbf789385.1707217097.git.bristot@kernel.org Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Bill Wendling <morbo@google.com> Cc: Justin Stitt <justinstitt@google.com> Cc: Donald Zickus <dzickus@redhat.com> Fixes: b1696371d865 ("rtla: Helper functions for rtla") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* tools/rtla: Fix clang warning about mount_point var sizeDaniel Bristot de Oliveira2024-02-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | clang is reporting this warning: $ make HOSTCC=clang CC=clang LLVM_IAS=1 [...] clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS $(pkg-config --cflags libtracefs) -c -o src/utils.o src/utils.c src/utils.c:548:66: warning: 'fscanf' may overflow; destination buffer in argument 3 has size 1024, but the corresponding specifier may require size 1025 [-Wfortify-source] 548 | while (fscanf(fp, "%*s %" STR(MAX_PATH) "s %99s %*s %*d %*d\n", mount_point, type) == 2) { | ^ Increase mount_point variable size to MAX_PATH+1 to avoid the overflow. Link: https://lkml.kernel.org/r/1b46712e93a2f4153909514a36016959dcc4021c.1707217097.git.bristot@kernel.org Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Bill Wendling <morbo@google.com> Cc: Justin Stitt <justinstitt@google.com> Cc: Donald Zickus <dzickus@redhat.com> Fixes: a957cbc02531 ("rtla: Add -C cgroup support") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* tools/rtla: Fix uninitialized bucket/data->bucket_size warningDaniel Bristot de Oliveira2024-02-122-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When compiling rtla with clang, I am getting the following warnings: $ make HOSTCC=clang CC=clang LLVM_IAS=1 [..] clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS $(pkg-config --cflags libtracefs) -c -o src/osnoise_hist.o src/osnoise_hist.c src/osnoise_hist.c:138:6: warning: variable 'bucket' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] 138 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~ src/osnoise_hist.c:149:6: note: uninitialized use occurs here 149 | if (bucket < entries) | ^~~~~~ src/osnoise_hist.c:138:2: note: remove the 'if' if its condition is always true 138 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~~~~~~ 139 | bucket = duration / data->bucket_size; src/osnoise_hist.c:132:12: note: initialize the variable 'bucket' to silence this warning 132 | int bucket; | ^ | = 0 1 warning generated. [...] clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS $(pkg-config --cflags libtracefs) -c -o src/timerlat_hist.o src/timerlat_hist.c src/timerlat_hist.c:181:6: warning: variable 'bucket' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] 181 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~ src/timerlat_hist.c:204:6: note: uninitialized use occurs here 204 | if (bucket < entries) | ^~~~~~ src/timerlat_hist.c:181:2: note: remove the 'if' if its condition is always true 181 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~~~~~~ 182 | bucket = latency / data->bucket_size; src/timerlat_hist.c:175:12: note: initialize the variable 'bucket' to silence this warning 175 | int bucket; | ^ | = 0 1 warning generated. This is a legit warning, but data->bucket_size is always > 0 (see timerlat_hist_parse_args()), so the if is not necessary. Remove the unneeded if (data->bucket_size) to avoid the warning. Link: https://lkml.kernel.org/r/6e1b1665cd99042ae705b3e0fc410858c4c42346.1707217097.git.bristot@kernel.org Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Bill Wendling <morbo@google.com> Cc: Justin Stitt <justinstitt@google.com> Cc: Donald Zickus <dzickus@redhat.com> Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* tools/rtla: Fix Makefile compiler options for clangDaniel Bristot de Oliveira2024-02-121-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following errors are showing up when compiling rtla with clang: $ make HOSTCC=clang CC=clang LLVM_IAS=1 [...] clang -O -g -DVERSION=\"6.8.0-rc1\" -flto=auto -ffat-lto-objects -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -Wno-maybe-uninitialized $(pkg-config --cflags libtracefs) -c -o src/utils.o src/utils.c clang: warning: optimization flag '-ffat-lto-objects' is not supported [-Wignored-optimization-argument] warning: unknown warning option '-Wno-maybe-uninitialized'; did you mean '-Wno-uninitialized'? [-Wunknown-warning-option] 1 warning generated. clang -o rtla -ggdb src/osnoise.o src/osnoise_hist.o src/osnoise_top.o src/rtla.o src/timerlat_aa.o src/timerlat.o src/timerlat_hist.o src/timerlat_top.o src/timerlat_u.o src/trace.o src/utils.o $(pkg-config --libs libtracefs) src/osnoise.o: file not recognized: file format not recognized clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [Makefile:110: rtla] Error 1 Solve these issues by: - removing -ffat-lto-objects and -Wno-maybe-uninitialized if using clang - informing the linker about -flto=auto Link: https://lore.kernel.org/linux-trace-kernel/567ac1b94effc228ce9a0225b9df7232a9b35b55.1707217097.git.bristot@kernel.org Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Bill Wendling <morbo@google.com> Cc: Justin Stitt <justinstitt@google.com> Fixes: 1a7b22ab15eb ("tools/rtla: Build with EXTRA_{C,LD}FLAGS") Suggested-by: Donald Zickus <dzickus@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla: Fix uninitialized variable foundColin Ian King2023-10-301-1/+1
| | | | | | | | | | | | Variable found is not being initialized, in the case where the desired mount is not found the variable contains garbage. Fix this by initializing it to zero. Link: https://lore.kernel.org/all/20230727150117.627730-1-colin.i.king@gmail.com/ Fixes: a957cbc02531 ("rtla: Add -C cgroup support") Signed-off-by: Colin Ian King <colin.i.king@gmail.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat: Do not stop user-space if a cpu is offlineDaniel Bristot de Oliveira2023-09-221-2/+4
| | | | | | | | | | | | | | | | | | If no CPU list is passed, timerlat in user-space will dispatch one thread per sysconf(_SC_NPROCESSORS_CONF). However, not all CPU might be available, for instance, if HT is disabled. Currently, rtla timerlat is stopping the session if an user-space thread cannot set affinity to a CPU, or if a running user-space thread is killed. However, this is too restrictive. So, reduce the error to a debug message, and rtla timerlat run as long as there is at least one user-space thread alive. Link: https://lore.kernel.org/lkml/59cf2c882900ab7de91c6ee33b382ac7fa6b4ed0.1694781909.git.bristot@kernel.org Fixes: cdca4f4e5e8e ("rtla/timerlat_top: Add timerlat user-space support") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat_aa: Fix previous IRQ delay for IRQs that happens after thread ↵Daniel Bristot de Oliveira2023-09-121-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | sample timerlat auto-analysis takes note of all IRQs, before or after the execution of the timerlat thread. Because we cannot go backward in the trace (we will fix it when moving to trace-cmd lib?), timerlat aa take note of the last IRQ execution in the waiting for the IRQ state, and then print it if it is executed after the expected timer IRQ starting time. After the thread sample, the timerlat starts recording the next IRQs as "previous" irq for the next occurrence. However, if an IRQ happens after the thread measurement but before the tracing stops, it is classified as a previous IRQ. That is not wrong, as it can be "previous" for the subsequent activation. What is wrong is considering it as a potential source for the last activation. Ignore the IRQ interference that happens after the IRQ starting time for now. A future improvement for timerlat can be either keeping a list of previous IRQ execution or using the trace-cmd library. Still, it requires further investigation - it is a new feature. Link: https://lore.kernel.org/lkml/a44a3f5c801dcc697bacf7325b65d4a5b0460537.1691162043.git.bristot@kernel.org Fixes: 27e348b221f6 ("rtla/timerlat: Add auto-analysis core") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat_aa: Fix negative IRQ delayDaniel Bristot de Oliveira2023-09-121-1/+17
| | | | | | | | | | | | | | | | | | | | When estimating the IRQ timer delay, we are dealing with two different clock sources: the external clock source that timerlat uses as a reference and the clock used by the tracer. There are also two moments: the time reading the clock and the timer in which the event is placed in the buffer (the trace event timestamp). If the processor is slow or there is some hardware noise, the difference between the timestamp and the external clock, read can be longer than the IRQ handler delay, resulting in a negative time. If so, set IRQ to start delay as 0. In the end, it is less near-zero and relevant then the noise. Link: https://lore.kernel.org/lkml/a066fb667c7136d86dcddb3c7ccd72587db3e7c7.1691162043.git.bristot@kernel.org Fixes: 27e348b221f6 ("rtla/timerlat: Add auto-analysis core") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat_aa: Zero thread sum after every sample analysisDaniel Bristot de Oliveira2023-09-121-0/+1
| | | | | | | | | | | | | The thread thread_thread_sum accounts for thread interference during a single activation. It was not being zeroed, so it was accumulating thread interference over all activations. It was not that visible when timerlat was the highest priority. Link: https://lore.kernel.org/lkml/97bff55b0141f2d01b47d9450a5672fde147b89a.1691162043.git.bristot@kernel.org Fixes: 27e348b221f6 ("rtla/timerlat: Add auto-analysis core") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
* rtla/timerlat_hist: Add timerlat user-space supportDaniel Bristot de Oliveira2023-06-131-11/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the support for running timerlat threads in user-space. In this mode, enabled with -u/--user-threads, timerlat dispatches user-space processes that will loop in the timerlat_fd, measuring the overhead for going to user-space and then returning to the kernel - in addition to the existing measurements. Here is one example of the tool's output with -u enabled: $ sudo timerlat hist -u -c 1-3 -d 600 # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:10:01 Index IRQ-001 Thr-001 Usr-001 IRQ-002 Thr-002 Usr-002 IRQ-003 Thr-003 Usr-003 0 477555 0 0 425287 0 0 474357 0 0 1 122385 7998 0 174616 1921 0 125412 3138 0 2 47 587376 492150 89 594717 447830 147 593463 454872 3 11 2549 101930 7 2682 145580 64 2530 138680 4 3 1954 2833 1 463 4917 11 548 4656 5 0 60 1037 0 138 1117 6 179 1130 6 0 26 1837 0 38 277 1 76 339 7 0 15 143 0 28 147 2 37 156 8 0 10 23 0 11 75 0 12 80 9 0 7 17 0 0 26 0 11 42 10 0 2 11 0 0 18 0 2 20 11 0 0 7 0 1 8 0 2 12 12 0 0 6 0 1 4 0 2 8 13 0 1 3 0 0 0 0 0 1 14 0 1 0 0 0 1 0 0 2 15 0 1 0 0 0 0 0 0 2 16 0 1 2 0 0 0 0 0 0 17 0 0 1 0 0 0 0 0 0 19 0 0 1 0 0 0 0 0 0 over: 0 0 0 0 0 0 0 0 0 count: 600001 600001 600001 600000 600000 600000 600000 600000 600000 min: 0 1 2 0 1 2 0 1 2 avg: 0 1 2 0 2 2 0 2 2 max: 4 16 19 4 12 14 7 12 15 The tuning setup like -p or -C work for the user-space threads as well. Link: https://lkml.kernel.org/r/b6a042d55003c4a67ff7dce28d96044b7044f00d.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/timerlat_top: Add timerlat user-space supportDaniel Bristot de Oliveira2023-06-137-6/+474
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the support for running timerlat threads in user-space. In this mode, enabled with -u/--user-threads, timerlat dispatches user-space processes that will loop in the timerlat_fd, measuring the overhead for going to user-space and then returning to the kernel - in addition to the existing measurements. Here is one example of the tool's output with -u enabled: $ sudo timerlat top -u -d 600 -q Timer Latency 0 00:10:01 | IRQ Timer Latency (us) | Thread Timer Latency (us) | Ret user Timer Latency (us) CPU COUNT | cur min avg max | cur min avg max | cur min avg max 0 #600001 | 0 0 0 3 | 2 1 2 9 | 3 2 3 15 1 #600001 | 0 0 0 2 | 2 1 2 13 | 2 2 3 18 2 #600001 | 0 0 0 10 | 2 1 2 16 | 3 2 3 20 3 #600001 | 0 0 0 7 | 2 1 2 10 | 3 2 3 11 4 #600000 | 0 0 0 16 | 2 1 2 41 | 3 2 3 58 5 #600000 | 0 0 0 3 | 2 1 2 10 | 3 2 3 13 6 #600000 | 0 0 0 5 | 2 1 2 7 | 3 2 3 10 7 #600000 | 0 0 0 1 | 2 1 2 7 | 3 2 3 10 The tuning setup like -p or -C work for the user-space threads as well. Link: https://lkml.kernel.org/r/758ad2292a0a1d884138d08219e1a0f572d257a2.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/hwnoise: Reduce runtime to 75%Daniel Bristot de Oliveira2023-06-131-1/+7
| | | | | | | | | | | | | | | | | | osnoise runs 100% of time by default. It makes sense because osnoise is preemptive. hwnoise checks preemption once a second, so it reduces system progress. Reduce runtime to 75% to avoid problems by default. I added a Fixes as it might avoid problems for first time users as it lands on distros. Link: https://lkml.kernel.org/r/af0b7113ffc00031b9af4bb40ef5889a27dadf8c.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Fixes: 1f428356c38d ("rtla: Add hwnoise tool") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-18 13:12:14 +0100