• Live demo.
• A "Feel Good" paper about reflection.
• Example of using reflection in an industrial setting - Java applications.
• SILK is a Scheme dialect in Java.
• Reflection benefits Both the Scheme and Java sides of SILK.
• Integrating Scheme with Java required generic functions.
• SILK as a compile time, and runtime scripting language.
• Reflection reveals meta helix issues:
  • Java security, Paranoid Quiz.
  • Limits of introspection on SILK.

• 1965 - L. Peter Deutch wrote a Lisp for the second PDP-1.
• 1970's - BBN Lisp renamed InterLisp.
• 1980's - Brain drain to Xerox PARC (including Danny Bobrow)
• 1982+ - Use of Lisp machines and Flavors.
• 1985+ - Using PCL -> CLOS standard
• 1990+ - Lisp and Scheme used in some projects.
• 1998+ - Began using SILK
• 1999 - Widening use of SILK in real applications.
• 1999 - Rusty Bobrow, Danny's brother uses SILK.

• Java

• Java Reflection

• Scheme

• Statically typed - variables are typed.
• C-like syntax
• Programs are made up of classes.
• Single inheritance of implementation (extends)
• Multiple inheritance of interfaces (implements).
  • All the overhead but none of the benefits of multiple inheritance.
• A class definition can contains (members):
  • Other (inner) class definitions
  • Static and instance fields
  • Static and instance methods
• Instance method - "this" distinquished argument.
• Static method - no distinquished argument.
• Primititive types, boolean, int, double, ... have wrapper classes.

• Packages java.lang and java.lang.reflect provide metaclasses that reflect a class and its components.

• Construct instance of existing classes.

• Invoke methods

• Metaclasses are final (can't be subclassed).

package elf;
import java.lang.reflect.*;
import java.util.Hashtable;

public class Reflect {
  public static void main(String[] args) {
    try {
      // Hashtable ht = new Hashtable(10);
      Constructor c = Hashtable.class.getConstructor
	(new Class[] { Integer.TYPE });
      Hashtable ht = (Hashtable) c.newInstance
	(new Object[] { new Integer(10) });

      // ht.put("Three", new Integer(3));
      Method m = Hashtable.class.getMethod
	("put", new Class[] { Object.class, Object.class });
      m.invoke(ht, new Object[] { "Three", new Integer(3) });

      System.out.println(ht);  // Prints: {Three=3}
    } catch (Exception e) { e.printStackTrace(); }}}

Scheme       Java
boolean      Boolean
symbol       silk.Symbol
char         Character
vector       Object[]
pair         silk.Pair
procedure    silk.Procedure
exact        Integer
inexact      Double
string       char[]
inputport    silk.InputPort
outputport   java.io.PrintWriter

• Compact - 50 page language spec, including denotation semantics.

• Uniform syntax: (function arg1 arg2 ...)

  
  ; Scheme                          Java
  (put ht "one" "first")            ht.put("one", "first")
  (+ (* a b) c)                     a*b + c
  

• A procedure can return another procedure - composition of behavior.

• A procedure can take any number of arguments - natural grouping, one liners.

• User defined syntax - minilanguages.

• SILK - "Scheme In about 50 K"

• Initial version by Peter Norvig - 20 hours, 650 lines.

• Eventually passed standard r4rstest test

• Tim Hickey provided JLIB graphic library

• Used in several introductory computer courses at Brandeis University.

• Silk 2.0 compiles syntactic expressions into Code objects.

• Current version has generic functions.

(constructor CLASSNAME ARGTYPE1 ...)
(method METHODNAME CLASSNAME ARGTYPE1 ...)

Example using BigInteger package: is 12345678987654321 probably prime?

> (define isProbablePrime
    (method "isProbablePrime" "java.math.BigInteger"
             "int"))
isProbablePrime
> (define BigInteger 
    (constructor "java.math.BigInteger"
                 "java.lang.String"))
BigInteger
> (isProbablePrime (BigInteger "12345678987654321") 10)
#f
> 

Try demo D2.

> (define class (method "forName" "java.lang.Class" 
                        "java.lang.String"))
class
> (define HT (class "java.util.Hashtable"))
HT
> HT
class java.util.Hashtable
> (define get-field 
    (method "getField" "java.lang.Class" "java.lang.String"))
get-field
> (define get-field-value 
    (method "get" "java.lang.reflect.Field" "java.lang.Object"))
get-field-value
>(define (get-static-value class field-name)
   (get-field-value (get-field class field-name) '())) ; '() is Java null.
get-static-value
> (get-static-value (class "java.lang.Void") "TYPE") ; In Java: Void.TYPE
void
>

1. Each method and constructor must be mentioned separately.
2. Procedures must be carefully named apart to avoid name conflicts.

   There are many potential conflicts, such as:

• Methods from different classes with same name:

  
  java.lang.reflect.Field.get(Object)
            java.util.Map.get(Object)
  

• Static and instance methods with the same name:

  
  static java.lang.reflect.Modifier.toString(int)
                    java.lang.Object.toString()
  

• overloaded methods:

  
  java.lang.StringBuffer.append() - has 10 methods.
  

Use Runtime type checks to choose the right method:

(define get
  (let ((hashtable-class (class "java.util.Hashtable"))
        (hashtable-get-method 
         (method "get" "java.util.Hashtable" "java.lang.Object"))
        (field-class (class "java.lang.reflect.Field"))
        (field-get-method 
         (method "get" "java.lang.reflect.Field" "java.lang.Object"))
        (isInstance 
         (method "isInstance" "java.lang.Class" "java.lang.Object")))
    (lambda (table key)
      ((cond
        ((isInstance hashtable-class table) hashtable-get-method)
        ((isInstance field-class table) field-get-method)
        (else (lambda (table key) (error "no method for " table))))
       table key))))

• The global variable Hashtable.class has the value of the class Hashtable.
• Each public instance method applicable to Hashtable is made a generic function.
  • This includes inherited methods, such as clone(), and toString().
  • Names that conflict with Scheme names are suffixed with "#":
    load#, substring#, length#, apply#, list#, append#, and print#

• Static methods are named Class.method.

• A generic constructor function: (Hashtable ...)

• Each public static final field is treated as a constant named Class.field.

Try demo D3.

> (define ht (Hashtable 20))
ht
> ht
{}
> (put ht 'clone 1)
()
> ht
{clone=1}
> (put ht 'zone 2)
()
> ht
{clone=1, zone=2}
> (get ht 'clone)
1
>

A generic function can have several methods:

> get
{silk.Generic get[3]}
> (for-each print (methods get))
{silk.InstanceMethod Object Map.get(Object)}
{silk.InstanceMethod Object silk.GlobalEnv.get(Symbol)}
{silk.InstanceMethod Object Field.get(Object)}
#t
>

> (import "java.lang.Float")
importing java.lang.Float in 81 ms.
#t
> (Float.parseFloat "17.42")
17.42
>

(import "java.lang.Double") (import "java.awt.Color")
(import "java.awt.Button")  (import "java.awt.TextField")
(import "jlib.EasyWin")

(define (test1)
  ;; Construct a test window.
  (let ((win (EasyWin "test1.scm" this-interpreter))
        (g (TextField "1" 20))
        (x (TextField "16" 20))
        (go (Button "Iterate")))
    (define (action e)           ; Define call back.
      (setText g
               (toString (f (Double.valueOf (getText g))
                            (Double.valueOf (getText x))))))
    (define (f g x)              ; Newton's method.
      (/ (+ g (/ x g)) 2.0))
    (resize win 200 200)         ; Size the window.
    (add win g)                  ; Add the components.
    (add win x)
    (add win go)
    (addActionCallback win action)
    (setBackground win (Color 200 200 255))
    (setVisible win #t)))

(test1)                          ; Try it out.

Object
  Procedure
    apply(Pair, Engine)
  Generic
    addMethod(GenericMethod)
    findMethod(GenericMethod)
    computeDiscriminator()

  GenericMethod
    match(GenericMehod)
    isApplicable(Pair)
    moreApplicableMethod(GenericMethod)

public abstract class Procedure extends SchemeUtils {
  // Apply the Procedure to a list of arguments.
  public abstract Object apply(Pair args, Engine eng);
}

public class Generic extends Procedure {
  // Add a method to the generic.
  public void addMethod(GenericMethod m); 
}

public abstract class GenericMethod extends Procedure {
  // Two GenericMethod's match if they have equal lists of parameterTypes.
  public boolean match(GenericMethod x);

  // Is the method applicable to the list of arguments?
  public boolean isApplicable(Pair args);

  // Returns the method that is more applicable.
  public GenericMethod moreApplicableMethod (GenericMethod m2);
}

Currently, four types of GenericMethod

• ConstructorMethod

• StaticMethod

• InstanceMethod

• ProcedureMethod

• Follow Java semantics closely,

• but, choose method at runtime

• Some ambiguity remains - see below.

;;; After importing java.lang.File and javax.swing.JFrame:
> list#
{silk.Generic list#[7]}
> (for-each print (methods list#))
{InstanceMethod String[] File.list(FilenameFilter)}
{InstanceMethod String[] File.list()}                    ****
{InstanceMethod void Component.list(PrintStream, int)}
{InstanceMethod void Component.list(PrintWriter, int)}
{InstanceMethod void Component.list()}
{InstanceMethod void Component.list(PrintWriter)}
{InstanceMethod void Component.list(PrintStream)}

• Null arguments provide no type information.
• Scheme uses only Integer and Double - must convert to get other types:

  ;;; java.lang.Hashtable(in, float)
  (Hashtable 10 (Float 0.75))
  

• Once Java returns a Float or Long, Scheme deals with it fine.
• Scheme strings and symbols must be mapped:

  Scheme type  Implementation type  Method parameter type Conversion
  string       char[]               String                char[] -> String
  symbol       silk.Symbol          String                Symbol -> String
  

• This leads to ambiguity in Stringbuffer.append():

  
  > (import "java.lang.StringBuffer")
  importing java.lang.StringBuffer in 70 ms.
  #t
  > append#
  {silk.Generic append#[10]}
  > (for-each print (methods append#))
  {InstanceMethod StringBuffer.append(char[], int, int)}
  {InstanceMethod StringBuffer.append(char[])}   ; ***
  {InstanceMethod StringBuffer.append(boolean)}
  {InstanceMethod StringBuffer.append(String)}   ; ***
  {InstanceMethod StringBuffer.append(Object)}
  {InstanceMethod StringBuffer.append(char)}
  {InstanceMethod StringBuffer.append(long)}
  {InstanceMethod StringBuffer.append(int)}
  {InstanceMethod StringBuffer.append(float)}
  {InstanceMethod StringBuffer.append(double)}
  

• SILK resolves this in favor of String.

• Let Java do most of the work of method lookup.

• Store only the most general Java method with a given signature.

• Example:

  
  > (methods toString)
  ({InstanceMethod String Object.toString()})
  > 
  

Statistics from 494 classes reachable from javax.swing.Jframe:

Count What
 494 classes
 458 public classes
  52 Exception classes
 176 static most general methods
2759 instance most general methods.
2935 total most general methods.

  Count Methods Cumulative % and examples
    110       1 82.1%
     15       2 93.3%
      6       3
      1       4 createPackedRaster
      1       5 getKeyStroke
      1       9 valueOf

  Count Methods Cumulative % and examples
   1058       1  69.4%
    255       2  86.1%
     75       3  91.0%
     39       4
     24       5
     23       6
     17       7
     10       8
      6       9
    ...     ...
      1      17 get
      1      18 contains
      3      20 clone insert print
      1      22 println
      1      24 remove
      1      36 add

• Optimize the few method case.
• Choose discriminator state as each method is added.
• Common Lisp makes this choice dynamically.

1 NOMETHODS

No methods, an error occurs if invoked.

2 ONEMETHOD

Simply invoke the method.

3 TWOMETHODINSTANCEFIXED

Two instance methods, same number of arguments. Check the first argument of first method.
If appliable, apply it, otherwise apply the second method.
This works because the types of the first arguments are disjoint.

4 TWOMETHODNOTFIXED

Two methods of any type with different numbers of arguments.
Choose the method based on the number of arguments.

5 GENERAL

Most general lookup.

• So, lookup is often little more than a switch jump and a subclass test or two.
• For cases where an error would occur, we simply invoke the wrong method and let Java signal the error.
• Most of the cost is still in crossing the Scheme/Java frontier.
  • Argument list to array.
  • Argument type conversion as above.
  • Return value conversion - #t #f

(import "java.util.Iterator") (import "java.util.Collection")
(import "java.util.Map")      (import "java.util.Vector")
(import "java.util.Hashtable")

(define-method iterate ((items java.util.Iterator) action)
  (if (hasNext items)
      (begin (action (next items))
             (iterate items action))))

(define-method iterate ((items java.util.Collection) action)
  (iterate (iterator items) action))

(define-method iterate ((items java.util.Map) action)
  (iterate (entrySet items) action))

(define-method iterate ((items silk.Pair) action)
  (action (car items))
  (let ((items (cdr items)))
    (if (pair? items) (iterate items action))))

(define-method iterate ((items java.lang.Object[]) action)
  (let loop ((i 0) 
             (L (vector-length items)))
    (if (< i L) (begin (action (vector-ref items i))
                       (loop (+ i 1) L)))))

> (define h (Hashtable 50))
h
> (put h 'fred 3)
()
> (put h 'mary 4)
()
> (iterate h print)
fred=3
mary=4
()

• Scheme methods treated like Java methods - fixed number of arguments.

• No (call-next-method).

• You can't do this in Java.

• Clearly SILK can be used as an embedded scripting and prototyping language.

• It can also be used as a compile time scripting language.

• Scheme declarative minilanguage -> compiles -> C or Java

• Use Java reflection to generate new classes from old ones.

• Used by Java beans to generate "glue code".

To trace a method, Hashtable.put():

• Subclass Hashtable
• Provide a put() method that prints trace information.
• Use the reflected method as a prototype.

> m2
public synchronized Object Hashtable.put(Object, Object)
> (emit (gen-trace-method m2))
public synchronized java.lang.Object
    put(java.lang.Object a1, java.lang.Object a0) {
  if(trace) Trace.enter(this + ".put(" + a1 + ", " + a0 + "}");
  java.lang.Object result = super.put(a1, a0);
  if(trace) Trace.exit(result);
  return result;
  }
#t

• (emit) formats a list structure like a Java program.
• Scheme "backquote macro" is used to construct lists that look like Java code.
• (map-args) is like (map) but provides a "," separator

(define-method gen-trace-method (m java.lang.reflect.Method)
  `(,(gen-method-signature m) { ,(gen-trace-method-body m) }))
  
(define-method gen-method-signature (m java.lang.reflect.Method)
  `(,(Modifier.toString (getModifiers m)) 
    ,(method-return-type m)
    ,(getName m) "(" 
    ,(map-args (lambda (arg) `(,(arg-type arg) ,(arg-name arg)))
     "," (arg-types m))
    ")"))
        
(define-method gen-trace-method-body (m java.lang.reflect.Method)
  `(if "(" trace ")" 
       Trace.enter "(" this + ,(quotify "." (getName m) "(")
       ,(map-args (lambda (arg) `(+ ,(arg-name arg)))
                  "+ \", \"" (arg-types m))
       + ,(quotify "}") ")" ";"
       ,(method-return-type m) result = super. 
       ,(getName m) "("
       ,(map-args arg-name "," (arg-types m))
       ")" ";"
       if "(" trace ")" "Trace.exit(result)" ";"
       return result ";"))

(define-method method-return-type (m java.lang.reflect.Method)
  (getName (getReturnType m)))

(define arg-type car)            ; Accessors
(define arg-name cadr)
(define (arg-types m)            ; method -> ((type name) ...)
  (let ((c 0))
    (map* (lambda (a) (list (getName a) (make-arg-name (set! c (+ c 1)))))
     (getParameterTypes m))))

(define (make-arg-name c)        ; (make-arg-name 1) -> "a1"
  (string->symbol (string-append "a" (number->string c))))

(define (quotify . args)         ; (quotify 'fred 3) -> "\"fred3\""
  (let ((double-quote "\""))
    (apply string-append double-quote (append args (list double-quote)))))

(import "java.io.File")         (import "java.lang.String")
(import "sun.tools.javac.Main") (import "java.lang.System")

(define (compile-file file)
  ;; Compile the *.java file, file using the current CLASSPATH.
  (let* ((file (if (string? file) file (toString file)))
	 (as (java-vector String.class
			  "-deprecation"
			  "-classpath"
			  (System.getProperty "java.class.path")
			  file))
	 (main (Main (get-static System.class 'out) 'silkc)))
    (compile main as)))

To compile the generated file:

> (compile-file "elf/TracedHashtable.java")
#t
>

• Skj
  • Like SILK
  • Focuses on being a Java scripting language.
  • Uses (invoke) and syntax.
• Jaja
  • Very object oriented implementation.
  • Based on Lisp in Small Pieces book
• Kawa
  • Very ambitious Scheme.
  • Compiles to Java byte code.

• Gregor Kiczales introduced the idea of a meta helix.

• When a metalevel extension are implemented in a base class. The extension is visible through reflection.

• Basically, you see implementation details you shouldn't.

• Common Lisp, Tiny CLOS, and Java have this problem.

• The solution is to keep the extension separate so the two view, base and extended, are not conflated.

• Here are some interesting examples of this in Java and SILK.

package elf;
/**
   The class Holder holds a secret int value.

   Once created, it will not reveal its secret to anyone.  Trust Java!
   However, isHappy() can be used to check if the secret is above 25.

   Holder has other private methods that are of no concern to us.

   QUIZ:  Can you write a class that can view or alter the secret? **/
public final class Holder {

  private int secret;

  public Holder(int secret) { this.secret = secret;}

  public final boolean isHappy() { return secret > 25; }

  private final Object booster() { 
    return new Object() {
      private final void boost() { secret++; }}; }

  public static boolean isHolder(Object x) {
    return x.getClass() == Holder.class; }
  // ...
}

> (describe (class "elf.Holder"))
public final class elf.Holder extends Object
HashCode: 1491376
ClassLoader: sun.misc.Launcher$AppClassLoader@4d3343
Name: elf.Holder
isArray: #f
Constructors:
  public elf.Holder(int)

Fields:
  private int secret
  static Class class$elf$Holder        ***  

Methods: 
  static int access$0(elf.Holder)      ***  
  static void access$1(elf.Holder,int) ***  
  static Class class$(String)          ***  
  private final Object booster()
  public final boolean isHappy()
  public static boolean isHolder(Object)

#f

package elf;

public class Sneaky
{
  public static void main(String[] args) {

    // Create a new Holder with a "secret" value
    Holder h = new Holder(42);

    // Retrieve secret value and print it
    System.out.println("i = " + Holder.access$0(h));	
  }
}

(define-method describe ((x java.lang.Object))
  (show "an instance of " (getName (getClass x)))
  (describe-fields x (getClass x)))

(define (describe-fields x superclass)
  (let ((fs (getDeclaredFields superclass)))
    (AccessibleObject.setAccessible fs #t) ; Make them all accessible.
       (for-each* (lambda (f)		   ; Not static fields.
		    (if (not (Modifier.isStatic (getModifiers f)))
			(describe-field f x)))
		  fs)
       (let ((superclass (getSuperclass superclass)))
	 (if (not (null? superclass)) (describe-fields x superclass)))))

(define (describe-field f x) (show "  " (getName f)": " (get f x)))

(define (show . items)
  (apply displays items)
  (newline))

(define (displays . items) (for-each display items))

> (describe h)
an instance of java.util.Hashtable
  table: #(Fred=3 () () () () () () () () Mary=4)
  count: 2
  threshold: 7
  loadFactor: 0.75
  modCount: 2
  keySet: ()
  entrySet: ()
  values: ()
()
> (describe 'hello)
an instance of silk.Symbol  ; Not turned into a String.
  name: hello
  globalIndex: -1
()
> (describe "hello")   ; "hello" is really a char[]!
an instance of java.lang.String
  value: hello         ; really a char[]
  offset: 0
  count: 5
()

Do what?

• Describe an object.

• Generate trace code.

How do you identify the object?

  public static void main(String[] args) {
    Object x = Magic.parse(args[0]);
    Describe.describe(x);
  }

You have to write a specialized main() or Magic.parse() for everything.

Even an introspective object browser would have limitations if it isn't scriptable.

This is why scripting languages are powerful.

• Never need to write a main() again.
• No need to write specialized input parsing code.
• Testing effort minimized.

• Many Scheme dialects
  • Easy to implement.
  • Optimization techniques well known.
  • Scheme is useful on its own
  • or embedded: Guile, STK

• Beckman - Scripting languages are inevitable - separation of concerns.

• Bentley - Little languages.

• Little languages become big - TCL, Visual Basic.

• Scheme is already complete.

• AWT and Swing GUI layout - separate GUI design from behavior.

• Complex processing task construction - maintains dependencies.

• Object input, user preferences - full language power.

• Java knowledgable Make - java.rmi.Remote implies rmic must be run.

• Run anyware scripting. - NT cmd.exe script can't sleep.

• Reflection based code generator - tiny.

• No compile Java prototyping - functioning spec, vs functional spec.
  • GCMonitor Try demo D8.
  • XML object serialization protocol

• Scheme benefits from Java objects.

• Java reflection is limited, but very useful.

• Reflective programming still abnormal for Java programmers - "Smoke and Mirrors".

• Reflective programming is easy in Scheme.

• In Java, types get in the way.

• SILK provides a view into Java that few people have seen.

• SILK's implementation can make better use of reflection to become smaller and faster.

• Method dispatch can be made "slightly dynamic" to provide faster dispatch.

• Goal: Download OpenMap and get running in 1/2 hour.

• Goal: Get your data displayed in 2 hours.

• Tiny example from Allan Doyle's talk - data integration and timeliness

• Interaction with a live BBN Java application.

• Current data - airlift out of ETAR (source GTN).

• Enrich data - ICAO -> location (like Addept)

• Feature mapping - quantity -> color.

• Fix bugs in data.

• Quick prototype - reveals data issues.