Scigraph - Scientific graphs

Jeffrey P. Morrill 617-873-4561

Contents

1.  What is Scigraph?

2.  Compiling and Loading the System

3.  Examples of Making and Displaying Graphs
3.1.  Running the Demo
3.2.  Graphing a Sequence of XY Pairs
3.3.  Graphing Multidimensional Data

4.  Graphical User Interface
4.1.  Common Graph Commands
4.2.  Common Annotation Commands
4.3.  Common Dataset Commands

5.  Programmer's Interface
5.1.  Dataset Classes
5.2.  Graph Classes
5.3.  Commonly Used Functions
5.4.  Dataset Protocols
5.5.  Hashtable Dataset Example
5.6.  Modifying Dataset Display Behavior
5.7.  Coordinate Systems

6.  Acknowledgements

7.  Copyright Notice



1.  What is Scigraph?

Scigraph is object oriented Common Lisp software for graphing numeric data.

Scigraph provides two groups of objects for plotting data, "graphs" and
"datasets".  The default behavior of these objects is to handle all the
details, including axis limits, data styles, legend position, and axis
ticks/labels/precision.  Pointer sensitivity provides users with
interactive control over all plot options.  There is also full programmatic
control over these options.

A graph can be thought of as a container for one or more datasets.  The
datasets are displayed on the interior of the graph, while the graph itself
provides labels, axes, and other annotations.

Several dataset classes are provided.  The easiest to use simply
requires a sequence of X,Y pairs.  Such a dataset can be displayed as a
line, a set of scattered points, or a bar graph.  Other dataset classes can
display equations, histograms, or contour data.  Other user objects can be
displayed as datasets if the user provides the dataset protocol described in
section 5.

Graphs support textual annotations.  The graph title, axis labels, and
legend are annotations.  These can be moved with the pointer, and their
character styles can be changed.  Annotations also allow the user to label
a graph with the x,y values of particular data points, or to compute
statistics for a region of data that the user lassoes with the pointer.

Scigraph requires Common Lisp and has been tested on Symbolics, Allegro,
and Lucid Lisp environments under Dynamic Windows and CLIM (version 0.9
or later).  Graphs may be displayed on either color or monochrome screens.  

This document begins by showing examples of how to create and display a
simple graph.  Then user interface operations such as editing, zooming, and
annotating are discussed.  Finally, portions of the software are documented
for programmers who wish to embed Scigraph in other applications.  The file
DEMO-FRAME.LISP gives examples of making and displaying graphs.



2.  Compiling and loading the system.

First you must compile and load the DWIM system.  Do this
by compiling and loading the file dwim/load-dwim.lisp.

Second, compile and load the file scigraph/load-scigraph.lisp.

When loading each of these files for the first time, you will need to
create a directory for the binary files to be placed in.  The loading
process will error if this directory does not exist, but there should be a
debugger proceed option to allow one to resume after the directory is
created.

Examples of Making and Displaying Graphs

These examples are for tutorial introduction to the facilities of 
Scigraph.  Here and elsewhere, the package is assumed to be GRAPH.

(in-package :graph)


3.1.  Running the demo.

The following makes a noisy sine wave dataset, an equation dataset, and
graphs them in a popup window (see demo-frame.lisp for its source code):

(make-demo-frame)

The following makes a simple bar graph:

(make-bar-demo)

This writes a graph to a postscript file:

(save-sample-graph)


3.2.  Graphing a Sequence of XY Pairs

As a simple example, suppose your data takes the form of a list of x-y 
pairs.  The following example shows how to make a graph.  

(setq raw-data '((0 0) (1 -1.5) (2 3) (3 22) (4 8) (5 22)))

(setq dataset (make-instance 'graph-data
			     :symbologies '(:line-symbol)
			     :data raw-data))

(setq graph   (make-instance 'annotated-graph
			     :title "Some Data"
			     :x-label "Fortnights"
			     :y-label "Furlongs"))

(add-dataset graph dataset)

The value of GRAPH is a graph object that is ready to be displayed.
There are several ways to display it.  If you want to pop up a window
showing only the specified graph, try:

(view-graphs (list graph))

If you want to display the graph on a specific stream, with some
specified position and size, try:

(stream-set-cursor-position* stream x y)
(display-graph graph :width width :height height :stream stream)

If you want a column of graphs (or several columns of graphs)
use FILL-WINDOW-WITH-GRAPHS.

The result is a graph of the raw data drawn on the specified stream, 
whose upper-left corner begins at the current cursor position and whose 
width and height are specified in pixels.  The graph has a title, axis 
labels, a legend, and of course the data itself.  Various parts of the 
graph are pointer sensitive, and moving the pointer around shows this.  
Click right over the center of the graph to get a menu of operations
appropriate to the graph as a whole.  Click right over the graph title 
(or any other pointer sensitive part to the graph) to get a menu of 
operations appropriate to it.

The purpose of the dataset object is to understand the structure of the 
raw data and be capable of mapping over the x-y pairs.  The default 
behavior is to assume that data points are lists of length 2, and that 
the raw data is a list of such points.  However, applications may 
override this default behavior to accomodate virtually any data 
structure.  

The default display behavior of a dataset is to use one or more
symbologies.  A "symbology" is the name of a way to draw a data point.
Available symbologies are: :SCATTER, :LINE, :LINE-SYMBOL, :STEP, :BAR.
Within the :SCATTER symbology, users may pick symbols such as :DIAMOND and
:CIRCLE, filled or unfilled, in some color.

The purpose of the graph object is to take care of coordinate
transformations, clipping, zooming, textual annotations, and the various
options relating to borders and labels.  Users may display several datasets
on a single graph using the ADD-DATASET function repeatedly.  Initially,
the graph is scaled such that all the data can be seen and such that it
fills the graph; this is called autoscaling.  Autoscaling may be turned off
to enable explicit control of axis limits.  Alternatively, datasets may
selectively provide information used in autoscaling.


3.3.  Graphing Multidimensional Data

Here is a more complex example.  In this example, individual datums are not
simply x, y pairs, but rather represent feature vectors with many
dimensions.  The application must provide accessor functions for extracting
the X and Y values from the datum.  The dimensions displayed on the graph
may be changed simply by changing the accessor functions.

(setq raw-data '(	(0 0 0 0) 
			(1 -1.5 2 4.5) 
			(2 3 4 5) 
			(3 22 11 6) 
			(4 8 12 17) 
			(5 22 19 15)))

(setq dataset (make-instance 'multidimensional-data
			     :symbologies '(:scatter)
			     :data raw-data)

(setq graph (make-instance 'graph-with-reselectable-axes))

(add-dataset graph dataset)

(set-axes graph #'first #'third)  

(view-graphs (list graph))   ; graph of 1st vs. 3rd

(set-axes graph #'second #'fourth)

(view-graphs (list graph))   ; redisplays using axes 2nd vs. 4th


4.  Graphical User Interface

4.1.  Common Graph Commands

Graph commands are generally invoked using the pointer, by clicking 
right on the graph.  This section documents those commands.  The 
commands are placed in a command table called GRAPH, and applications 
that use graphs should include this table in the command table for the
application.


Crosshairs

This is useful for finding the exact coordinates of a point on the 
graph.  Crosshairs are placed on the graph, and they track the pointer 
until some pointer button is pressed.  Along each axis, the exact x and 
y values of the crosshairs are displayed and constantly updated.


Zoom In/Out

This is useful for displaying a closeup of a particular region of the 
graph.  Using crosshairs, two opposing corners of a box are chosen.  
When done, the graph is redrawn, with the axis limits set to the edges 
of the chosen box.  One may zoom in arbitrarily many times.  Each zoom 
command is recorded on a stack, such that "Zoom Out" pops the zoom stack 
and returns to the immediately preceding axis limits.  The "Zoom Out" 
command disappears as a choice in the graph menu once the stack is 
emptied.


 Edit Graph Borders & Labels

This is useful for controlling many of the detailed aspects of the 
display.  A window pops up and displays the current attributes of the 
graph.  Users modify attributes by clicking the pointer on the 
applicable field and using the keyboard to change the value.  Click on 
OK to finish editing.  Click on CANCEL to exit the dialog box without
applying the changes.

Display Legend.  The Legend displays an icon for each dataset in the 
margin area below the graph.  Each dataset in the legend is pointer-
sensitive, and clicking on the icon provides a menu of dataset 
operations.  This field controls whether or not the Legend is displayed.

Autoscaling Options.  Autoscaling sets the initial axis edges such that 
all the data can be seen and it fills the display.  These options can be 
used to selectively turn off the autoscale behavior.

Display Grid.  A grid is superimposed on the graph.

Labels and Borders.  Set axis options such as labels, tick spacing, and 
tick numbering.

Axis Limits.  Explicitly enter the x and y limits to use as the edges of 
the graph.


Redraw Graph

This command redisplays the display.  It is useful particularly during
debugging when something has gone wrong with previous display 
operations.  


Add Free Text

Annotations allow a user to add text to graphs interactively.  An
annotation is a piece of text on a graph that can be moved or edited.
There are several types of annotations, for example:
Plain Text
One or several lines of text that you enter and place 
anywhere.

Plain Text + Pointer
Like plain text but a line is drawn from the text 
pointing to something you select with the pointer.

Identify Data Point

You are prompted to select a data point using the pointer.  An annotation
of the type "Plain Text + Pointer" will be added identifying the X and Y
coordinates of the point.
Identify Data Region

You are prompted to select a closed polygon by iteratively clicking the
pointer.  Finally, click on the initial point to close the polygon.
You will then be prompted to choose one or more statistical measures.
An annotation will be added identifying the statistical values for all
data points that fall within the closed region.

4.2 Common Annotation Commands

Once an annotation has been added to the graph, there are several commands
available.  Graph titles, axis labels and legends are all annotations.

Move

Drag the annotation to another location on the graph.  On a text
annotation, one may only move the text.  On a point annotation, the 
pointer may be moved independently of the text to point to something 
else.  On an "Region" annotation, the points of the polygon region
are independently moveable.  Note that "Move" is the default operation
on annotations and may be invoked immediately by clicking the pointer
left on the annotation to drag it.

Change Style

Select a character style from a menu.  A variety of sizes 
are provided in combinations of bold, italic, and normal fonts.

Delete

Permanently removes the annotation.

Edit Annotation Text

Edit the text of the annotation.

4.3.   Common Dataset Commands

To get the menu of dataset commands, you must find a dataset object
to click the pointer on.  By default, datasets are displayed on
the "legend".  The legend is a rectangular annotation that lists
the name and color of each dataset on the graph.  Scigraph attempts
to automatically position the legend in a place that won't obscure
any of the data.  Click right on the name of the dataset to get a
menu of the available commands.

If the legend is not displayed, then [Change Data Symbols] may
be executed by first executing [Edit Graph Borders & Labels]
(by selecting this command from the click-right menu of the graph).
In that dialog is a list of dataset names.  Click left on the dataset
name you are interested in.


Change Data Symbols

This is useful for controlling many of the detailed aspects of how
the data points are displayed.  A dialog window pops up and displays the
current attributes of the dataset.  Users modify attributes by clicking
the pointer on the applicable field and using the keyboard to change
the value.  Click on OK to finish editing.  Click on CANCEL to exit
without applying the changes.

Plotting Styles
  There are five plotting styles: bar, step, line,
line-symbol, and scatter.  Any or all of them may be selected at one time.
Once a style has been selected, additional options may be controlled.  For
example, on a scatter graph one may choose the symbol to use for data points;
for a line graph, one may choose the line style.

Dither
  This is useful particularly for low resolution data where many
data points occur at the same pixel position, making it difficult to see
how many points there are there.  Dither adds a small random component to the X
and/or Y value to spread out the points.  The number is chosen in the units
of the X and Y axes.

Sensitivity of data points
  This controls the pointer sensitivity of the
individual data points.  When they are "sensitive," a datum that is under the 
pointer will get highlighted, and clicking right on it will provide a 
menu of operations appropriate to that datum.  For large datasets
(> 1,000 points), it may be prudent to turn off sensitivity
for efficiency (a special optimized drawing routine is used for
this case).

 Hide Dataset

This redraws the graph without showing this dataset.  It is most useful
when there are many datasets and you decide you don't want to see one
of them anymore.  When there are hidden datasets, the graph has an
extra command called [Reveal Hidden Datasets] for un-hiding them.

5.  Programmer's Interface

Scigraph exports its functionality from the GRAPH package.  (Scigraph 
also includes internal packages called DWIM and TOOL that users should not 
ordinarily refer to.)  A package definition that uses Scigraph might 
look like the following:

(defpackage my-package (:use common-lisp graph))


Scigraph puts its commands into the :GRAPH command table as well as the 
:GLOBAL command table.  Programs that incorporate graphs should include 
one of these command tables in the command table of the program.  In 
CLIM, this is done as follows:
(define-command-table 'my-command-table :inherit-from (:graph))


5.1.  Dataset Classes

There are several classes of datasets that Scigraph provides.  

GRAPH-DATA
.  The simplest type of dataset.  The raw data must be a 
sequence (list or array) whose elements are each a list of length 2.  
The X value is the first element of the list, the Y value is the second.

MULTIDIMENSIONAL-DATA
 A subclass of GRAPH-DATA.  The raw data must be 
a sequence (list or array), but elements of the sequence may be any 
arbitrary object.  The X value of the object is found by evaluating
(FUNCALL (X-ACCESSOR dataset) object), the Y value by (FUNCALL (Y-ACCESSOR 
dataset) object).  Therefore the dimensions that are seen on the graph 
are controlled setting and resetting the X and Y accessors.  Use (SET-
AXES dataset x-accessor y-accessor) to reset them.

EQUATION-DATA
 A sublcass of GRAPH-DATA.  A dataset where the raw
data is not an enumerated sequence of data points, but rather an equation.
For example:

(make-instance 	'equation-data
			:variable 'x
			:equation '(* (sin (* a x)) (sin (* b x)))
			:parameters '((a 2) (b 3))
			:min 0
			:max 10
			:increment .1)

The parameters, a and b, may be edited to change the appearance of the dataset.

HISTOGRAM-DATA 
A subclass of GRAPH-DATA.  Used for displaying a histogram
of a set of values.  The set of values must be of the form of a list of
numbers.  Below is an example:

(setq set-of-values '(1 3 5 3 9 0 2 3 ...))
(make-instance 'histogram-data :sample-data set-of-values)


>The values are collected into bins and displayed as a bar graph.
If you want 20 bins (and thus 20 bars), supply the keyword
:BIN-COUNT 20 to make-instance.  Alternatively, you may specify
the keyword :BIN-SIZE to specify the width of each bin.  Neither
of these keywords is required.

5.2.  Graph Classes

Scigraph provides several classes of graphs.

GRAPH
  A simple graph, providing most of the default behavior.  This is a
lowest-common-denominator graph, not fit for human consumption.

ANNOTATED-GRAPH 
A subclass of GRAPH.  This supplies the graph with a title
and axis labels that are annotations.  

GRAPH-WITH-RESELECTABLE-AXES
 A subclass of ANNOTATED-GRAPH.  Designed to
be used in combination with datasets of type MULTIDIMENSIONAL-DATA.

5.3.  Commonly Used Functions

(VIEW-GRAPHS graphs &key columns autoscale reverse-video title
                             left bottom width height)

Pops up a window display of the specified graphs.  Left, bottom, width,
and height specify the position and size of the window in pixels relative
to the upper left of the screen.  The list of graphs is displayed in
the specified number of columns, using as many rows as seem necessary.

(FILL-WINDOW-WITH-GRAPHS graphs &key columns autoscale stream reverse-video)

Displays one or more columns of graphs.  This is what VIEW-GRAPHS uses
to do the actual display.

(DISPLAY-GRAPHS graphs &key stream width height)

Displays one column of graphs, beginning at the current cursor position.
This is used by FILL-WINDOW-WITH-GRAPHS.

(DISPLAY-GRAPH graph &key stream width height)>
Displays one graph, beginning at the current cursor position.
This is used by DISPLAY-GRAPHS.
(SAVE-POSTSCRIPT-GRAPH graph filename &key width height)

Saves the graph in encapsulated postscript format.

(REFRESH graph stream)

Erases and then redisplays a displayed graph.

(ADD-DATASET graph dataset)

The next time the graph is displayed, the dataset will appear on the 
graph.  Many datasets may be displayed on a single graph by invoking 
this function repeatedly.


5.4.  Dataset Protocols

Interaction between graphs and datasets is through several protocols.
These protocols have subprotocols and new dataset behavior may be defined
by extending either the protocol or subprotocol levels.  Some protocols are
optional, meaning that default methods that do the right thing (usually
nothing) are provided.
The following protocols are required:

Display Data protocol:
  DISPLAY-DATA                  - How to display the dataset.
    MAP-DATA                    - Map function over each dataum of the dataset.
    MAP-DATA-XY                 - Map function over the x,y values of each datum.
      DATUM-POSITION            - Return the position of a datum.
      DATUM-DISPLAYER           - Function to display a datum.
        DATUM-STYLE-DISPLAYER   - Function to display datum in a named style.

Autoscale protocol:
  AUTO-SCALE-LIMITS             - Provide information on scale requirements.

The following protocols are optional:

Presentation protocol:
  PRESENT-SELF-P                - Am I presentable?
    GRAPH-PRESENT-INFERIORS-P   - Should I present my inferiors?
    GRAPH-PRESENTATION-TYPE     - Type of presentation.

Legend Protocol
  SHOW-LEGEND                   - Show legend for this dataset?
    DISPLAY-LEGEND-DATASET      - How to display the dataset for this dataset.
    ...                           See legend.lisp for details.

Popup Accept Protocol: 
  POPUP-ACCEPT                  - How to edit an object.
    POPUP-ACCEPTABLE            - Is object editable?
    POPUP-ACCEPT-ITEMS          - Items to accept.
    POPUP-ACCEPT-LABEL          - A label for the editor window.

Graph Label protocol:
  TITLE                         - Suggest a title for the graph.
  X-LABEL                       - Suggest an x axis label for the graph.
  Y-LABEL                       - Suggest a y axis label for the graph.


5.5  Hashtable Dataset Example

When using raw data that does not follow the simple  sequence-of-lists 
structure described above in an example, applications must provide two 
methods to use for mapping over the data:  MAP-DATA and DATUM-POSITION.  
Below is an example for the case where the raw data is not a sequence, 
but rather a hash table.  In addition, individual datums are not simple 
lists, but rather arrays.
(defclass hash-data-mixin () )

(defmethod map-data ((self hash-data-mixin) (function t) (data t))
   "Apply the function to each datum"
    (maphash #'(lambda (key value)
                 (declare (ignore key))
                 (funcall function value))
             data))

(defmethod datum-position ((self hash-data-mixin) (datum array))
    "Get the actual X and Y values to plot."
    (values (aref datum 0) (aref datum 1)))

The application program may also control the presentation type of the 
displayed datums.  The default presentation type is simply EXPRESSION.
(defmethod datum-presentation-type ((self hash-data-mixin) (datum t))
    'hash-datum)


5.6.  Modifying Dataset Display Behavior

Many interesting types of datasets can be defined as subclasses of
GRAPH-DATA by modifying the display protocol in some way.  For example, see
the source code for EQUATION-DATA and CONTOUR-DATA.  The protocol of
dataset display is as follows:

(DISPLAY-DATA dataset stream graph)

The top-level call for a dataset to display itself.  The default
behavior is to compute a datum display function and map it over
each datum.

 
map over each data point (using MAP-DATA)
 
Convert a datum to x,y coordinates (using DATUM-POSITION)
 
Convert from x,y to stream pixel position
 
Call the display function (computed using DATUM-DISPLAYER)

(DATUM-DISPLAYER dataset graph)

Returns a function whose arguments are (STREAM U V DATUM).
Calling the function once displays one datum.  Default behavior
is to construct a displayer that calls a set of datum displayers,
one displayer for each symbology of the datum.

(DATUM-STYLE-DISPLAYER dataset graph style-type)

Returns a function whose arguments are (STREAM U V DATUM).
Calling the function once displays one datum in the given
style.  Predefined styles include :LINE, :SCATTER, and :BAR.
The body of a displayer is typically one or more calls to
simple stream draw operations.  The displayer for :SCATTER 
might be defined (for a clim stream) as follows:
    #'(lambda (STREAM U V DATUM)
	(declare (ignore datum))
        (multiple-value-setq (u v) (uv-to-screen stream u v))
	(draw-point* stream u v))

5.7.  Coordinate systems

There are three coordinate systems that a programmer might be
concerned about.
  XY Coordinates.  This is the coordinates of your data.

  UV Coordinates.  This is like the stream coordinates
    except that the positive direction of the vertical axis
    is upward instead of downward.
  SCREEN Coordinates.  This is the coordinates of the stream.
    It probably should be called stream coordinates.  

There are various functions for converting between coordinate
systems.  For example XY-TO-UV and UV-TO-STREAM.


6.  Acknowledgements

Many people contributed to Scigraph over the years.  These are the ones we
can still remember: Patrick Love, Douglas Spindler, and other at Alcoa
Technical Center.  Albert Boulanger, Dan Cerys, Nicheal Cramer, Jeff
Mattson, Mike Thome, Brian Wilson, and
      others of BBN.
Thanks to Sheldon S. Ball  for porting this system
to MCL.  Thanks to David L. Westbrook for
porting this system to Harlequin Lispworks and for piles of bug fixes.


7.  Copyright Notice

Copyright (c) 1987-1993 by BBN Systems and Technologies,
A Division of Bolt, Beranek and Newman Inc.
All rights reserved.

Permission to use, copy, modify and distribute this software and its
documentation is hereby granted without fee, provided that the above
copyright notice of BBN Systems and Technologies, this paragraph and the
one following appear in all copies and in supporting documentation, and
that the name Bolt Beranek and Newman Inc. not be used in advertising or
publicity pertaining to distribution of the software without specific,
written prior permission. Any distribution of this software or derivative
works must comply with all applicable United States export control laws.

BBN makes no representation about the suitability of this software for any
purposes.  It is provided "AS IS", without express or implied warranties
including (but not limited to) all implied warranties of merchantability
and fitness for a particular purpose, and notwithstanding any other
provision contained herein.  In no event shall BBN be liable for any
special, indirect or consequential damages whatsoever resulting from loss
of use, data or profits, whether in an action of contract, negligence or
other tortuous action, arising out of or in connection with the use or
performance of this software, even if BBN Systems and Technologies is
advised of the possiblity of such damages.

To: RShapiro@bbn.com
Cc: Kanderson@bbn.com
From: Ken Anderson 
Subject: histograms
Date: Thu, 17 Oct 1996 17:07:14 -0400
Sender: kanderso@bbn.com
Content-Type: text
X-UIDL: e70be37f6766209ee3dbac2d14f7614c
Status: RO


;;; This makes a simple histogram of gaussian data.

(in-package graph)

(defun draw-it ()
  (let* ((d (make-instance 'histogram-data
	      :sample-data (gaussian-random-sample 500) 
	      :pattern t
	      :color +red+))
	 (g (make-instance 'annotated-graph
	      :x-tick-numbering :each
	      :y-tick-numbering :each
	      )))
    (add-dataset g d)
    (view-graphs (list g))))

For scatter plots when you make a dataset use :symbologies (list :scatter)
:data-symbol :circle.

See equation.lisp you can set things up to switch quickly from on x vs y to
another:

;;; The following dataset (MULTIDIMENSIONAL-DATA) and graph class
;;; (GRAPH-WITH-RESELECTABLE-AXES) go together.  They are for the case where
;;; individual datums are not really xy pairs but more like feature vectors with many
;;; dimensions.  You provide two accessor functions, one each for the x and y axes,
;;; using the SET-AXES function.  Those two axes will then be displayed.  SET-AXES
;;; may then be used to change the dimensions being displayed without having to make
;;; a new graph.