Scigraph - Scientific graphs

Ken Anderson 617-873-3160

Jeffrey P. Morrill 617-873-4561


Contents

1. What is Scigraph? 2. Compiling and Loading the System 3. Examples of Making and Displaying Graphs 3.1. Running the Demo 3.2. Graphing a Sequence of XY Pairs 3.3. Graphing Multidimensional Data 4. Graphical User Interface 4.1. Common Graph Commands 4.2. Common Annotation Commands 4.3. Common Dataset Commands 5. Programmer's Interface 5.1. Dataset Classes 5.2. Graph Classes 5.3. Commonly Used Functions 5.4. Dataset Protocols 5.5. Hashtable Dataset Example 5.6. Modifying Dataset Display Behavior 5.7. Coordinate Systems 6. Acknowledgements 7. Copyright Notice

1. What is Scigraph?

Scigraph is object oriented Common Lisp software for graphing numeric data. Scigraph provides two groups of objects for plotting data, "graphs" and "datasets". The default behavior of these objects is to handle all the details, including axis limits, data styles, legend position, and axis ticks/labels/precision. Pointer sensitivity provides users with interactive control over all plot options. There is also full programmatic control over these options. A graph can be thought of as a container for one or more datasets. The datasets are displayed on the interior of the graph, while the graph itself provides labels, axes, and other annotations. Several dataset classes are provided. The easiest to use simply requires a sequence of X,Y pairs. Such a dataset can be displayed as a line, a set of scattered points, or a bar graph. Other dataset classes can display equations, histograms, or contour data. Other user objects can be displayed as datasets if the user provides the dataset protocol described in section 5. Graphs support textual annotations. The graph title, axis labels, and legend are annotations. These can be moved with the pointer, and their character styles can be changed. Annotations also allow the user to label a graph with the x,y values of particular data points, or to compute statistics for a region of data that the user lassoes with the pointer. Scigraph requires Common Lisp and has been tested on Symbolics, Allegro, and Lucid Lisp environments under Dynamic Windows and CLIM (version 0.9 or later). Graphs may be displayed on either color or monochrome screens. This document begins by showing examples of how to create and display a simple graph. Then user interface operations such as editing, zooming, and annotating are discussed. Finally, portions of the software are documented for programmers who wish to embed Scigraph in other applications. The file DEMO-FRAME.LISP gives examples of making and displaying graphs.

2. Compiling and loading the system.

First you must compile and load the DWIM system. Do this by compiling and loading the file dwim/load-dwim.lisp. Second, compile and load the file scigraph/load-scigraph.lisp. When loading each of these files for the first time, you will need to create a directory for the binary files to be placed in. The loading process will error if this directory does not exist, but there should be a debugger proceed option to allow one to resume after the directory is created.

Examples of Making and Displaying Graphs

These examples are for tutorial introduction to the facilities of Scigraph. Here and elsewhere, the package is assumed to be GRAPH. (in-package :graph)

3.1. Running the demo.

The following makes a noisy sine wave dataset, an equation dataset, and graphs them in a popup window (see demo-frame.lisp for its source code): (make-demo-frame) The following makes a simple bar graph: (make-bar-demo) This writes a graph to a postscript file: (save-sample-graph)

3.2. Graphing a Sequence of XY Pairs

As a simple example, suppose your data takes the form of a list of x-y pairs. The following example shows how to make a graph. (setq raw-data '((0 0) (1 -1.5) (2 3) (3 22) (4 8) (5 22))) (setq dataset (make-instance 'graph-data :symbologies '(:line-symbol) :data raw-data)) (setq graph (make-instance 'annotated-graph :title "Some Data" :x-label "Fortnights" :y-label "Furlongs")) (add-dataset graph dataset) The value of GRAPH is a graph object that is ready to be displayed. There are several ways to display it. If you want to pop up a window showing only the specified graph, try: (view-graphs (list graph)) If you want to display the graph on a specific stream, with some specified position and size, try: (stream-set-cursor-position* stream x y) (display-graph graph :width width :height height :stream stream) If you want a column of graphs (or several columns of graphs) use FILL-WINDOW-WITH-GRAPHS. The result is a graph of the raw data drawn on the specified stream, whose upper-left corner begins at the current cursor position and whose width and height are specified in pixels. The graph has a title, axis labels, a legend, and of course the data itself. Various parts of the graph are pointer sensitive, and moving the pointer around shows this. Click right over the center of the graph to get a menu of operations appropriate to the graph as a whole. Click right over the graph title (or any other pointer sensitive part to the graph) to get a menu of operations appropriate to it. The purpose of the dataset object is to understand the structure of the raw data and be capable of mapping over the x-y pairs. The default behavior is to assume that data points are lists of length 2, and that the raw data is a list of such points. However, applications may override this default behavior to accomodate virtually any data structure. The default display behavior of a dataset is to use one or more symbologies. A "symbology" is the name of a way to draw a data point. Available symbologies are: :SCATTER, :LINE, :LINE-SYMBOL, :STEP, :BAR. Within the :SCATTER symbology, users may pick symbols such as :DIAMOND and :CIRCLE, filled or unfilled, in some color. The purpose of the graph object is to take care of coordinate transformations, clipping, zooming, textual annotations, and the various options relating to borders and labels. Users may display several datasets on a single graph using the ADD-DATASET function repeatedly. Initially, the graph is scaled such that all the data can be seen and such that it fills the graph; this is called autoscaling. Autoscaling may be turned off to enable explicit control of axis limits. Alternatively, datasets may selectively provide information used in autoscaling.

3.3. Graphing Multidimensional Data

Here is a more complex example. In this example, individual datums are not simply x, y pairs, but rather represent feature vectors with many dimensions. The application must provide accessor functions for extracting the X and Y values from the datum. The dimensions displayed on the graph may be changed simply by changing the accessor functions. (setq raw-data '( (0 0 0 0) (1 -1.5 2 4.5) (2 3 4 5) (3 22 11 6) (4 8 12 17) (5 22 19 15))) (setq dataset (make-instance 'multidimensional-data :symbologies '(:scatter) :data raw-data) (setq graph (make-instance 'graph-with-reselectable-axes)) (add-dataset graph dataset) (set-axes graph #'first #'third) (view-graphs (list graph)) ; graph of 1st vs. 3rd (set-axes graph #'second #'fourth) (view-graphs (list graph)) ; redisplays using axes 2nd vs. 4th

4. Graphical User Interface

4.1. Common Graph Commands

Graph commands are generally invoked using the pointer, by clicking right on the graph. This section documents those commands. The commands are placed in a command table called GRAPH, and applications that use graphs should include this table in the command table for the application.
Crosshairs
This is useful for finding the exact coordinates of a point on the graph. Crosshairs are placed on the graph, and they track the pointer until some pointer button is pressed. Along each axis, the exact x and y values of the crosshairs are displayed and constantly updated.
Zoom In/Out
This is useful for displaying a closeup of a particular region of the graph. Using crosshairs, two opposing corners of a box are chosen. When done, the graph is redrawn, with the axis limits set to the edges of the chosen box. One may zoom in arbitrarily many times. Each zoom command is recorded on a stack, such that "Zoom Out" pops the zoom stack and returns to the immediately preceding axis limits. The "Zoom Out" command disappears as a choice in the graph menu once the stack is emptied.
Edit Graph Borders & Labels
This is useful for controlling many of the detailed aspects of the display. A window pops up and displays the current attributes of the graph. Users modify attributes by clicking the pointer on the applicable field and using the keyboard to change the value. Click on OK to finish editing. Click on CANCEL to exit the dialog box without applying the changes.

Display Legend. The Legend displays an icon for each dataset in the margin area below the graph. Each dataset in the legend is pointer- sensitive, and clicking on the icon provides a menu of dataset operations. This field controls whether or not the Legend is displayed.

Autoscaling Options. Autoscaling sets the initial axis edges such that all the data can be seen and it fills the display. These options can be used to selectively turn off the autoscale behavior.

Display Grid. A grid is superimposed on the graph.

Labels and Borders. Set axis options such as labels, tick spacing, and tick numbering.

Axis Limits. Explicitly enter the x and y limits to use as the edges of the graph.

Redraw Graph
This command redisplays the display. It is useful particularly during debugging when something has gone wrong with previous display operations.
Add Free Text
Annotations allow a user to add text to graphs interactively. An annotation is a piece of text on a graph that can be moved or edited. There are several types of annotations, for example:
Plain Text
One or several lines of text that you enter and place anywhere.
Plain Text + Pointer
Like plain text but a line is drawn from the text pointing to something you select with the pointer.
Identify Data Point
You are prompted to select a data point using the pointer. An annotation of the type "Plain Text + Pointer" will be added identifying the X and Y coordinates of the point.
Identify Data Region
You are prompted to select a closed polygon by iteratively clicking the pointer. Finally, click on the initial point to close the polygon. You will then be prompted to choose one or more statistical measures. An annotation will be added identifying the statistical values for all data points that fall within the closed region.

4.2 Common Annotation Commands

Once an annotation has been added to the graph, there are several commands available. Graph titles, axis labels and legends are all annotations.
Move
Drag the annotation to another location on the graph. On a text annotation, one may only move the text. On a point annotation, the pointer may be moved independently of the text to point to something else. On an "Region" annotation, the points of the polygon region are independently moveable. Note that "Move" is the default operation on annotations and may be invoked immediately by clicking the pointer left on the annotation to drag it.
Change Style
Select a character style from a menu. A variety of sizes are provided in combinations of bold, italic, and normal fonts.
Delete
Permanently removes the annotation.
Edit Annotation Text
Edit the text of the annotation.

4.3. Common Dataset Commands

To get the menu of dataset commands, you must find a dataset object to click the pointer on. By default, datasets are displayed on the "legend". The legend is a rectangular annotation that lists the name and color of each dataset on the graph. Scigraph attempts to automatically position the legend in a place that won't obscure any of the data. Click right on the name of the dataset to get a menu of the available commands. If the legend is not displayed, then [Change Data Symbols] may be executed by first executing [Edit Graph Borders & Labels] (by selecting this command from the click-right menu of the graph). In that dialog is a list of dataset names. Click left on the dataset name you are interested in.
Change Data Symbols
This is useful for controlling many of the detailed aspects of how the data points are displayed. A dialog window pops up and displays the current attributes of the dataset. Users modify attributes by clicking the pointer on the applicable field and using the keyboard to change the value. Click on OK to finish editing. Click on CANCEL to exit without applying the changes.
Plotting Styles
There are five plotting styles: bar, step, line, line-symbol, and scatter. Any or all of them may be selected at one time. Once a style has been selected, additional options may be controlled. For example, on a scatter graph one may choose the symbol to use for data points; for a line graph, one may choose the line style.
Dither
This is useful particularly for low resolution data where many data points occur at the same pixel position, making it difficult to see how many points there are there. Dither adds a small random component to the X and/or Y value to spread out the points. The number is chosen in the units of the X and Y axes.
Sensitivity of data points
This controls the pointer sensitivity of the individual data points. When they are "sensitive," a datum that is under the pointer will get highlighted, and clicking right on it will provide a menu of operations appropriate to that datum. For large datasets (> 1,000 points), it may be prudent to turn off sensitivity for efficiency (a special optimized drawing routine is used for this case).
Hide Dataset
This redraws the graph without showing this dataset. It is most useful when there are many datasets and you decide you don't want to see one of them anymore. When there are hidden datasets, the graph has an extra command called [Reveal Hidden Datasets] for un-hiding them.

5. Programmer's Interface

Scigraph exports its functionality from the GRAPH package. (Scigraph also includes internal packages called DWIM and TOOL that users should not ordinarily refer to.) A package definition that uses Scigraph might look like the following:

(defpackage my-package (:use common-lisp graph))

Scigraph puts its commands into the :GRAPH command table as well as the :GLOBAL command table. Programs that incorporate graphs should include one of these command tables in the command table of the program. In CLIM, this is done as follows:
(define-command-table 'my-command-table :inherit-from (:graph))

5.1. Dataset Classes

There are several classes of datasets that Scigraph provides.
GRAPH-DATA
. The simplest type of dataset. The raw data must be a sequence (list or array) whose elements are each a list of length 2. The X value is the first element of the list, the Y value is the second.
MULTIDIMENSIONAL-DATA
A subclass of GRAPH-DATA. The raw data must be a sequence (list or array), but elements of the sequence may be any arbitrary object. The X value of the object is found by evaluating (FUNCALL (X-ACCESSOR dataset) object), the Y value by (FUNCALL (Y-ACCESSOR dataset) object). Therefore the dimensions that are seen on the graph are controlled setting and resetting the X and Y accessors. Use (SET- AXES dataset x-accessor y-accessor) to reset them.
EQUATION-DATA
A sublcass of GRAPH-DATA. A dataset where the raw data is not an enumerated sequence of data points, but rather an equation. For example:

(make-instance 	'equation-data
			:variable 'x
			:equation '(* (sin (* a x)) (sin (* b x)))
			:parameters '((a 2) (b 3))
			:min 0
			:max 10
			:increment .1)
The parameters, a and b, may be edited to change the appearance of the dataset.
HISTOGRAM-DATA
A subclass of GRAPH-DATA. Used for displaying a histogram of a set of values. The set of values must be of the form of a list of numbers. Below is an example:

(setq set-of-values '(1 3 5 3 9 0 2 3 ...))
(make-instance 'histogram-data :sample-data set-of-values)


>The values are collected into bins and displayed as a bar graph.
If you want 20 bins (and thus 20 bars), supply the keyword
:BIN-COUNT 20 to make-instance.  Alternatively, you may specify
the keyword :BIN-SIZE to specify the width of each bin.  Neither
of these keywords is required.

5.2. Graph Classes

Scigraph provides several classes of graphs.
GRAPH
A simple graph, providing most of the default behavior. This is a lowest-common-denominator graph, not fit for human consumption.
ANNOTATED-GRAPH
A subclass of GRAPH. This supplies the graph with a title and axis labels that are annotations.
GRAPH-WITH-RESELECTABLE-AXES
A subclass of ANNOTATED-GRAPH. Designed to be used in combination with datasets of type MULTIDIMENSIONAL-DATA.

5.3. Commonly Used Functions

(VIEW-GRAPHS graphs &key columns autoscale reverse-video title left bottom width height)
Pops up a window display of the specified graphs. Left, bottom, width, and height specify the position and size of the window in pixels relative to the upper left of the screen. The list of graphs is displayed in the specified number of columns, using as many rows as seem necessary.
(FILL-WINDOW-WITH-GRAPHS graphs &key columns autoscale stream reverse-video)
Displays one or more columns of graphs. This is what VIEW-GRAPHS uses to do the actual display.
(DISPLAY-GRAPHS graphs &key stream width height)
Displays one column of graphs, beginning at the current cursor position. This is used by FILL-WINDOW-WITH-GRAPHS.
(DISPLAY-GRAPH graph &key stream width height)>
Displays one graph, beginning at the current cursor position. This is used by DISPLAY-GRAPHS.
(SAVE-POSTSCRIPT-GRAPH graph filename &key width height)
Saves the graph in encapsulated postscript format.
(REFRESH graph stream)
Erases and then redisplays a displayed graph.
(ADD-DATASET graph dataset)
The next time the graph is displayed, the dataset will appear on the graph. Many datasets may be displayed on a single graph by invoking this function repeatedly.

5.4. Dataset Protocols

Interaction between graphs and datasets is through several protocols. These protocols have subprotocols and new dataset behavior may be defined by extending either the protocol or subprotocol levels. Some protocols are optional, meaning that default methods that do the right thing (usually nothing) are provided.
The following protocols are required:

Display Data protocol:
  DISPLAY-DATA                  - How to display the dataset.
    MAP-DATA                    - Map function over each dataum of the dataset.
    MAP-DATA-XY                 - Map function over the x,y values of each datum.
      DATUM-POSITION            - Return the position of a datum.
      DATUM-DISPLAYER           - Function to display a datum.
        DATUM-STYLE-DISPLAYER   - Function to display datum in a named style.

Autoscale protocol:
  AUTO-SCALE-LIMITS             - Provide information on scale requirements.

The following protocols are optional:

Presentation protocol:
  PRESENT-SELF-P                - Am I presentable?
    GRAPH-PRESENT-INFERIORS-P   - Should I present my inferiors?
    GRAPH-PRESENTATION-TYPE     - Type of presentation.

Legend Protocol
  SHOW-LEGEND                   - Show legend for this dataset?
    DISPLAY-LEGEND-DATASET      - How to display the dataset for this dataset.
    ...                           See legend.lisp for details.

Popup Accept Protocol: 
  POPUP-ACCEPT                  - How to edit an object.
    POPUP-ACCEPTABLE            - Is object editable?
    POPUP-ACCEPT-ITEMS          - Items to accept.
    POPUP-ACCEPT-LABEL          - A label for the editor window.

Graph Label protocol:
  TITLE                         - Suggest a title for the graph.
  X-LABEL                       - Suggest an x axis label for the graph.
  Y-LABEL                       - Suggest a y axis label for the graph.

5.5 Hashtable Dataset Example

When using raw data that does not follow the simple sequence-of-lists structure described above in an example, applications must provide two methods to use for mapping over the data: MAP-DATA and DATUM-POSITION. Below is an example for the case where the raw data is not a sequence, but rather a hash table. In addition, individual datums are not simple lists, but rather arrays.
(defclass hash-data-mixin () )

(defmethod map-data ((self hash-data-mixin) (function t) (data t))
   "Apply the function to each datum"
    (maphash #'(lambda (key value)
                 (declare (ignore key))
                 (funcall function value))
             data))

(defmethod datum-position ((self hash-data-mixin) (datum array))
    "Get the actual X and Y values to plot."
    (values (aref datum 0) (aref datum 1)))
The application program may also control the presentation type of the displayed datums. The default presentation type is simply EXPRESSION.
(defmethod datum-presentation-type ((self hash-data-mixin) (datum t))
    'hash-datum)

5.6. Modifying Dataset Display Behavior

Many interesting types of datasets can be defined as subclasses of GRAPH-DATA by modifying the display protocol in some way. For example, see the source code for EQUATION-DATA and CONTOUR-DATA. The protocol of dataset display is as follows:
(DISPLAY-DATA dataset stream graph)
The top-level call for a dataset to display itself. The default behavior is to compute a datum display function and map it over each datum.
  1. map over each data point (using MAP-DATA)
  2. Convert a datum to x,y coordinates (using DATUM-POSITION)
  3. Convert from x,y to stream pixel position
  4. Call the display function (computed using DATUM-DISPLAYER)
(DATUM-DISPLAYER dataset graph)
Returns a function whose arguments are (STREAM U V DATUM). Calling the function once displays one datum. Default behavior is to construct a displayer that calls a set of datum displayers, one displayer for each symbology of the datum.
(DATUM-STYLE-DISPLAYER dataset graph style-type)
Returns a function whose arguments are (STREAM U V DATUM). Calling the function once displays one datum in the given style. Predefined styles include :LINE, :SCATTER, and :BAR. The body of a displayer is typically one or more calls to simple stream draw operations. The displayer for :SCATTER might be defined (for a clim stream) as follows:
    #'(lambda (STREAM U V DATUM)
	(declare (ignore datum))
        (multiple-value-setq (u v) (uv-to-screen stream u v))
	(draw-point* stream u v))

5.7. Coordinate systems

There are three coordinate systems that a programmer might be concerned about.
  XY Coordinates.  This is the coordinates of your data.

  UV Coordinates.  This is like the stream coordinates
    except that the positive direction of the vertical axis
    is upward instead of downward.
  SCREEN Coordinates.  This is the coordinates of the stream.
    It probably should be called stream coordinates.  
There are various functions for converting between coordinate systems. For example XY-TO-UV and UV-TO-STREAM.

6. Acknowledgements

Many people contributed to Scigraph over the years. These are the ones we can still remember: Patrick Love, Douglas Spindler, and other at Alcoa Technical Center. Albert Boulanger, Dan Cerys, Nicheal Cramer, Jeff Mattson, Mike Thome, Brian Wilson, and others of BBN. Thanks to Sheldon S. Ball for porting this system to MCL. Thanks to David L. Westbrook for porting this system to Harlequin Lispworks and for piles of bug fixes.

7. Copyright Notice

Copyright (c) 1987-1993 by BBN Systems and Technologies, A Division of Bolt, Beranek and Newman Inc. All rights reserved. Permission to use, copy, modify and distribute this software and its documentation is hereby granted without fee, provided that the above copyright notice of BBN Systems and Technologies, this paragraph and the one following appear in all copies and in supporting documentation, and that the name Bolt Beranek and Newman Inc. not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. Any distribution of this software or derivative works must comply with all applicable United States export control laws. BBN makes no representation about the suitability of this software for any purposes. It is provided "AS IS", without express or implied warranties including (but not limited to) all implied warranties of merchantability and fitness for a particular purpose, and notwithstanding any other provision contained herein. In no event shall BBN be liable for any special, indirect or consequential damages whatsoever resulting from loss of use, data or profits, whether in an action of contract, negligence or other tortuous action, arising out of or in connection with the use or performance of this software, even if BBN Systems and Technologies is advised of the possiblity of such damages.
To: RShapiro@bbn.com
Cc: Kanderson@bbn.com
From: Ken Anderson 
Subject: histograms
Date: Thu, 17 Oct 1996 17:07:14 -0400
Sender: kanderso@bbn.com
Content-Type: text
X-UIDL: e70be37f6766209ee3dbac2d14f7614c
Status: RO


;;; This makes a simple histogram of gaussian data.

(in-package graph)

(defun draw-it ()
  (let* ((d (make-instance 'histogram-data
	      :sample-data (gaussian-random-sample 500) 
	      :pattern t
	      :color +red+))
	 (g (make-instance 'annotated-graph
	      :x-tick-numbering :each
	      :y-tick-numbering :each
	      )))
    (add-dataset g d)
    (view-graphs (list g))))

For scatter plots when you make a dataset use :symbologies (list :scatter)
:data-symbol :circle.

See equation.lisp you can set things up to switch quickly from on x vs y to
another:

;;; The following dataset (MULTIDIMENSIONAL-DATA) and graph class
;;; (GRAPH-WITH-RESELECTABLE-AXES) go together.  They are for the case where
;;; individual datums are not really xy pairs but more like feature vectors with many
;;; dimensions.  You provide two accessor functions, one each for the x and y axes,
;;; using the SET-AXES function.  Those two axes will then be displayed.  SET-AXES
;;; may then be used to change the dimensions being displayed without having to make
;;; a new graph.


Ken Anderson
Last modified: Mon Oct 28 14:59:05 EST