Generating statistics

Concept Information

STATISTICS command

You can generate detailed statistics on numeric and datetime fields in a table. Statistics provide an overview of a table, and can highlight abnormalities in the data, which can guide your subsequent analysis.

When you generate statistics, in addition to the standard output options, Analytics automatically creates a number of system variables that contain the output results. For more information, see System variables created by Analytics commands.

The results of generating statistics are described in the table below.

Note

All the statistics are generated for numeric fields. Only a subset of the statistics are generated for datetime fields.

Several of the datetime statistics are displayed as serial datetime values. You can use a conversion function to convert a serial datetime value to a human-readable regular datetime value. For more information, see Serial datetimes.

Statistic name Numeric field Datetime field
Range The difference between the highest and lowest values The number of days between the most recent and oldest dates
Positive

  • The number of positive values

  • The total of all positive values

  • The average positive value

  • The number of datetime values

  • The total of all serial datetime values

    Not meaningful for dates, or time values that represent points in time. Meaningful for time values that represent amounts of time.

    For more information, see An amount of time versus a point in time.

  • The average datetime value
Negative

  • The number of negative values

  • The total of all negative values

  • The average negative value

 Not applicable, not generated
Zeros The number of zero values The number of blank dates
Totals

  • The total number of positive, negative, and zero values

  • The total of all positive, negative, and zero values

  • The average of all positive, negative, and zero values

 Not applicable, not generated
Abs Value The total of all values while disregarding the sign of the numbers Not applicable, not generated
Std Dev (optional) The standard deviation from the mean value

The standard deviation from the average datetime value

Not displayed when outputting statistics to screen, but captured in the STDDEVn system-generated output variable.
Median (optional) The median value

  • Odd-numbered sets of values: the middle value

  • Even-numbered sets of values: the average of the two values at the middle

 The median date

  • Odd-numbered sets of dates: the middle date

  • Even-numbered sets of dates: the average of the two dates at the middle
Q25 (optional) The first quartile value (lower quartile value)

  • The result is an interpolated value based on an Analytics algorithm

  • Produces the same result as the QUARTILE and QUARTILE.INC functions in Microsoft Excel

 The first quartile date (lower quartile date)

  • The result is an interpolated value based on an Analytics algorithm

  • Offset by 1 day from the result of the QUARTILE and QUARTILE.INC functions in Microsoft Excel

    For more information, see Serial datetimes.
Q75 (optional) The third quartile value (upper quartile value)

  • The result is an interpolated value based on an Analytics algorithm

  • Produces the same result as the QUARTILE and QUARTILE.INC functions in Microsoft Excel

 The third quartile date (upper quartile date)

  • The result is an interpolated value based on an Analytics algorithm

  • Offset by 1 day from the result of the QUARTILE and QUARTILE.INC functions in Microsoft Excel

    For more information, see Serial datetimes.
Mode (optional) The most frequently occurring value

  • Displays “N/A” if no value occurs more than once

  • In the event of a tie, displays the lowest value

 The most frequently occurring date

  • Displays “N/A” if no date occurs more than once

  • In the event of a tie, displays the oldest date
Highest

The five highest values

You can change the number of high values with the # of High/Low setting on the More tab in the Statistics dialog box.

The five most recent dates

You can change the number of recent dates with the # of High/Low setting on the More tab in the Statistics dialog box.
Lowest

The five lowest values

You can change the number of low values with the # of High/Low setting on the More tab in the Statistics dialog box.

The five oldest dates

You can change the number of oldest dates with the # of High/Low setting on the More tab in the Statistics dialog box.

Steps

You can generate descriptive statistics on numeric and datetime fields in a table.

  1. Select Analyze > Statistics.
  2. On the Main tab, do one of the following:

    Select the field(s) to generate statistics on from the Statistics On list.

    Click Statistics On to select the field(s), or to create an expression.

    The order in which you select the fields is the order in which the columns appear in the results.

  3. If there are records in the current view that you want to exclude from processing, enter a condition in the If text box, or click If to create an IF statement using the Expression Builder.

    Note

    The If condition is evaluated against only the records remaining in a table after any scope options have been applied (First, Next, While).

    The IF statement considers all records in the view and filters out those that do not meet the specified condition.

  4. If you want to calculate the standard deviation for the selected field or fields, select Std Deviation.
  5. If you want to calculate the median, mode, and first and third quartile values for the selected field or fields, select Median, Mode, Q25, Q75.

    Note

    Calculating these additional statistics requires additional computer memory. You may exceed your computer's memory and get an error message if you calculate the additional statistics for very large data sets.

  6. Click the Output tab.
  7. Select the appropriate output option in the To panel:
    • Screen – Select this option to display the results in the Analytics display area.

      Tip

      You can click any linked result value in the display area to drill down to the associated record or records in the source table.

      If the output table contains a large number of records, it is faster and more useful to save the results to a file than to display the results on the screen.

    • Print – Select this option to send the results to the default printer.
    • Graph – Select this option to create a graph of the results and display it in the Analytics display area.
    • File – Select this option to save or append the results to a text file. The file is saved outside Analytics.

    Note

    Output options that do not apply to a particular analytical operation are disabled.

  8. If you selected File as the output type, specify the following information in the As panel:
    • File Type – ASCII Text File or Unicode Text file (depending on which edition of Analytics you are using) is the only option. Saves the results to a new text file, or appends the results to an existing text file.
    • Name – Enter a file name in the Name text box. Or click Name and enter the file name, or select an existing file in the Save or Save File As dialog box to overwrite or append to the file. If Analytics prefills a file name, you can accept the prefilled name, or change it.

      You can also specify an absolute or relative file path, or navigate to a different folder, to save or append the file in a location other than the project location. For example: C:\Results\Output.txt or Results\Output.txt.

    • Local – Disabled and selected. Saving the file locally is the only option.
  9. Click the More tab.

  10. In the Scope panel, select the appropriate option:

    • All
    • First
    • Next
    • While
    Scope option Details
    All (Default) Specifies that all records in the view are processed.
    First Enter a number in the text box. Starts processing at the first record in the view and includes only the specified number of records.
    Next Enter a number in the text box. Starts processing at the currently selected record in the view and includes only the specified number of records. The actual record number must be selected in the leftmost column in the view, not data in the row.
    While

    Use a WHILE statement to limit the processing of records in the view based on a particular criterion or set of criteria.

    Enter a condition in the While text box, or click While to create a WHILE statement using the Expression Builder.

    A WHILE statement allows records in the view to be processed only while the specified condition evaluates to true. As soon as the condition evaluates to false, the processing terminates, and no further records are considered.

    You can use the While option in conjunction with the All, First, or Next options. Record processing stops as soon as one limit is reached.

    Note

    The First or Next options reference either the physical order or the indexed order of records in a table. First or Next disregard any filtering or quick sorting applied to a table view. However, output results of analytical operations respect any filtering.

    If a view is quick sorted, Next behaves like First.

  11. If you want to change the number of highest and lowest field values included in the results, enter the number in # of High/Low.
  12. If you selected File as the output type, and want to append the output results to the end of an existing text file, select Append To Existing File.
  13. Click OK.
  14. If the overwrite prompt appears, select the appropriate option.