Interacting with graphics

Purpose: We will explain how to make a Java program interact with the user's mouse.

A simple interactive program

Relevant Interfaces. Java has two built in interfaces which are designed to handle mouse interactions:
1. The MouseListener interface handles interactions with the mouse that don't involve motion of the mouse. The following functions make up this interface:
```
void mouseClicked(MouseEvent e);
void mouseEntered(MouseEvent e);
void mouseExited(MouseEvent e);
void mousePressed(MouseEvent e);
void mouseReleased(MouseEvent e);
```
2. The MouseMotionListener interface handles motion of the mouse. It allows users to drag objects around for instance. The following functions make up the interface:
```
void mouseDragged(MouseEvent e);
void mouseMoved(MouseEvent e);
```
3. A MouseEvent object is passed to each of these methods. This class contains all the information you could possibly want to know about the event such as the location of the mouse, which buttons were clicked or pressed, and more.
Program Idea. We will create a simple program which records where a user clicks and places a colored disk at any point which is clicked. The user will also be able to drag the disks around.

We will put all our graphics in a JPanel which we will extend to do what we want. Here is the beginning of the class definition:
```
public class InteractionDemo extends JPanel implements MouseListener, MouseMotionListener {

    // This list records the places the user clicks:
    private LinkedList<Point2D> click_locations=new LinkedList<Point2D>();

    private static final int R = 20; // Radii of disks

    public InteractionDemo() {
        setBackground(Color.WHITE);

        // Necessary for mouse interaction:
        addMouseListener(this);
        addMouseMotionListener(this);
    }
    
    ...
}
```
Comments:
- The class will implement the MouseListener and MouseMotionListener interfaces. This means the class will contain the methods mentioned on the last page.
- The addMouseListener(this) and addMouseMotionListener(this) tell the JPanel to call these methods when the user does something with the mouse. You can add
- You can implement these interfaces in other objects. Many people implement the interfaces using inner classes.
- The integer R represents the radius of our disks. It is a constant and can't be changed.

Recording clicks. The following is an implementation of the MouseListener and MouseMotionListener interfaces which just appends the location clicked to the click_locations list and forces the panel to be repainted.

public class InteractionDemo extends JPanel implements MouseListener, MouseMotionListener {

    private LinkedList<Point2D> click_locations=new LinkedList<Point2D>();

    ... constructor ...

    public void mouseClicked(MouseEvent me) {
        click_locations.addLast(me.getPoint());
        repaint();
    }

    public void mouseEntered(MouseEvent me) {}
    public void mouseExited(MouseEvent me) {}
    public void mousePressed(MouseEvent me) {}
    public void mouseReleased(MouseEvent me) {}
    public void mouseDragged(MouseEvent me) {}
    public void mouseMoved(MouseEvent me) {}

The paint routine: We iterate through the clicks and draw a disk about every point. We choose a coloring scheme to make the disks look nice.

    public void paintComponent(Graphics gfx) {
        super.paintComponent(gfx);
        Graphics2D g = (Graphics2D) gfx;
        g.setRenderingHint(RenderingHints.KEY_ANTIALIASING,
                RenderingHints.VALUE_ANTIALIAS_ON);

        g.setStroke(new BasicStroke(2)); // Draw curves which are 2 pixels wide.
        
        double hue = 0; // for picking different colors. 
        for (Point2D pt : click_locations) {
	    // The circle of radius R about pt.
            Ellipse2D e = new Ellipse2D.Double(pt.getX() - R, pt.getY() - R, 2 * R, 2 * R);
            g.setColor(Color.getHSBColor((float) hue, 1f, 1f));
            g.fill(e);
            g.setColor(Color.BLACK);
            g.draw(e);

            hue = hue + (Math.sqrt(5) - 1) / 2; // Change the hue by 
            hue = hue - Math.floor(hue);        // an irrational rotation.
        }
    }

Dragging. Dragging something consists of several steps. First the user presses a button. Then, she drags (moves the mouse while holding the mouse). Finally, she releases the button.

On a mouse press, our program will record if the mouse is located in a disk. If it is, we record the disk that contains the mouse location. As the mouse is dragged, we update the location of the disk. When the button is released, we stop collecting this information.

    boolean disk_selected=false;
    int selected_disk_index=0;
    Point2D last_mouse_location;

    public void mousePressed(MouseEvent me) {
        Iterator<Point2D> it=click_locations.descendingIterator();
        int index=click_locations.size()-1;
        while (it.hasNext()) { // Descend through the list of click locations.
            if (me.getPoint().distance(it.next())<R) {
                disk_selected=true;
                selected_disk_index=index;
                last_mouse_location=me.getPoint();
                break; // quit the while loop.
            }
            index = index-1;
        }
    }

    public void mouseReleased(MouseEvent me) {
        disk_selected=false;
    }

    public void mouseDragged(MouseEvent me) {
        if (disk_selected) {
            Point2D current_center=click_locations.get(selected_disk_index);
            Point2D new_center=new Point2D.Double(
                    current_center.getX()+me.getX()-last_mouse_location.getX(),
                    current_center.getY()+me.getY()-last_mouse_location.getY());
            click_locations.set(selected_disk_index, new_center);
            last_mouse_location=me.getPoint();
            repaint();
        }
    }

Main Method. Our panel also has a main method, which constructs a window an places our demo inside.

The source code can be read here: InteractionDemo.java.

Fractal Path Manipulator

Idea of the program:
We will create a new program to manipulate fractal paths generated using a substitution scheme. We will fix a single polygonal path, γ, joining zero to one in the complex plane. Then we get a fractal by iteratively substitution the path for the edges of the path.

The user will be able to interact with the program in various "modes". For instance, the user will be able to:
- move around the points making up γ.
- add or delete the vertices making up γ.
- zoom in to better examine the curve.
The user will be able to select the mode of interaction using the window's menus.

Once a mode is selected, we will display a brief message explaining how specifically the user should behave.

The Window.

The class FractalPathDisplay will extend JFrame and thus represent a window we can open. The window will display three things: a menu bar, a panel called a FractalPathPanel for drawing the fractal, and a status bar for providing small bits of information to the user.

The basic outline of the class is displayed below:

public class FractalPathDisplay extends JFrame {

    FractalPathPanel panel;
    JLabel status_bar;

    public FractalPathDisplay() {
        panel = new FractalPathPanel();        
	this.setLayout(new BorderLayout());
        add(panel, BorderLayout.CENTER); 
        
	// Our status bar is just a label meaning it can display text:
        status_bar=new JLabel(); 
        add(status_bar, BorderLayout.SOUTH);
	// Our panel will have a setStatusBar method, which lets the panel
        // manipulate the status bar.
        panel.setStatusBar(status_bar); 
        
	// setup the window
        setJMenuBar(buildMenuBar());
    }
    
    /** This function creates and returns the menu bar. */
    private JMenuBar buildMenuBar() { ... }

    /** Create a FractalPathDisplay window and display it. */
    public static void main(String[] args) {
        FractalPathDisplay display = new FractalPathDisplay();

        // Configure the window:
        display.setTitle("Fractal Path Manipulator"); // Set the window's title.
        display.setSize(640, 480); // Set the dimensions of the window
        display.setDefaultCloseOperation(EXIT_ON_CLOSE);

        display.setVisible(true);
    }
}

The Panel.

The FractalPathPanel is a pretty complicated class because it will handle all the interaction modes, draw the curve, and output to the status bar. But, it will not be too difficult, if we break down its jobs into pieces.

Setup. The class FractalPathPanel will extend JPanel and allow mouse interaction. Here is a minimal implementation satisfying these requirements:

public class FractalPathPanel extends JPanel implements MouseListener, MouseMotionListener {
    public FractalPathPanel() {}
        // Necessary for mouse interaction:
        addMouseListener(this);
        addMouseMotionListener(this);
    }

    // MouseListener:
    void mouseClicked(MouseEvent e){}
    void mouseEntered(MouseEvent e){}
    void mouseExited(MouseEvent e){}
    void mousePressed(MouseEvent e){}
    void mouseReleased(MouseEvent e){}

    // MouseMotionListener:
    void mouseDragged(MouseEvent e){}
    void mouseMoved(MouseEvent e){}
}

The status bar. The panel must be able to output text to the status bar. Here is the

The status bar. The panel must be able to output text to the status bar. The FractalPathDisplay will call a setStatusBar method to give the panel access to the status bar. Here is the implementation:

    private JLabel status_bar = null;

    /** Set the label to print status updates to. */
    public void setStatusBar(JLabel new_bar) {
        status_bar = new_bar;
        // This will update the status bar with the current mode of interaction:
        setInteractionMode(current_mode);
    }

    /** Set the text in the status_bar provided the bar exists. */
    private void postStatus(String status) {
        if (status_bar != null) {
            status_bar.setText(status);
        }
    }

Note that the initially status_bar is set to null until the status bar is set. We don't output information as long as status_bar is null.

The postStatus method is private, because we only will call it from within the class. (We don't want other classes to be able to update the status bar directly.)

The fractal curve. We will be displaying a fractal curve determined by a polygonal path gamma. Here is the code we use for achieving this modifying achieving this:

    private VertexPath gamma;            // Curve used to "seed" the substitution fractal
    private SubstitutionFractal fractal; // The fractal curve we display
    private int depth = 2;               // Depth to look at for the approximation
    private PolygonalPath approx_path;   // The approximation to the fractal curve we display

    public final void setGamma(PolygonalPath gamma) {
        this.gamma = new VertexPath(gamma);
        fractal = new SubstitutionFractal(gamma, gamma);
        approx_path = new PolygonalApproximation(fractal, depth);
        repaint();
    }

    public final void setDepth(int new_depth) {
        // Prevent depth from becoming negative, and ignore false changes.
        if ((new_depth >= 0) && (new_depth != depth)) {
            depth = new_depth;
            approx_path = new PolygonalApproximation(fractal, depth);
            postStatus("Fractal approximation depth set to " + depth + ".");
            repaint();
        }
    }

    /** Decrease the quality of the polygonal approximation. */
    public final void decreaseDepth() {
        setDepth(depth - 1);
    }

    /** Increase the quality of the polygonal approximation. */
    public final void increaseDepth() {
        setDepth(depth + 1);
    }

We also update the constructor to call setGamma to default to the path which generates the Koch snowflake.

The two coordinate systems. As before, we consider two coodinate systems, screen coordiantes and math coordinates. We convert between them using the AffineTransform class. We keep track of a rectangle which we guarantee to display. The affine transform is so that the box "just fits" into the window.

We have written helper functions for dealing with the coordinate change, and two functions for manipulating the box we guarantee to display.

    // We guarantee that this box will be displayed.
    private Rectangle2D display_box;
    // This affine transformation converts from math coordinates to screen coordinates.
    private AffineTransform current_transform = new AffineTransform();

    /** This function returns an affine transform which sends display_box into
     *  the rectangle representing screen coordinates for this panel. */
    private AffineTransform getTransform() { ... }
 
    /** Convert the Complex number z into screen coordinates. */
    private Point2D toScreenCoordinates(Complex z) { ... }
    }

    /** Convert the point from screen coordinates into math coordinates. */
    private Complex toMathCoordinates(Point2D p) { ... }

    /** Guarantee the whole curve is displayed in the window. */
    public final void fit() {
        display_box = PathUtil.boundingBox(approx_path);
	postStatus("Zooming to fit the fractal curve in the window.");
        repaint();
    }

    /** Rescale the box we are guaranteed to display by a multiplicative constant
     *  via a dilation fixing the provided complex number. This achieves zooming 
     *  toward and away from a point */
    public void scale(double scaling_constant, Complex fixed_point) { 
	// ... modify display_box appropriately ...
	repaint();
    }

To see the implementation of these methods, see FractalPathPanel.java.

The drawing routine. Our paintComponent method will just draw the curve. Later we will draw some additional stuff depending on the mode of interaction.

    public void paintComponent(Graphics gfx) {
        // This stuff is standard, and should be in any paintComponent method. 
        super.paintComponent(gfx);
        Graphics2D g = (Graphics2D) gfx;
        g.setRenderingHint(RenderingHints.KEY_ANTIALIASING,
                RenderingHints.VALUE_ANTIALIAS_ON);

        // Reset the transformation which converts from math coordinates to screen coordinates.
        current_transform = getTransform();

        // Draw in black, with lines of width equal to one pixel. 
        g.setColor(Color.BLACK);
        g.setStroke(new BasicStroke(1));
	// PathDrawing.drawPolygonalPath is a static method which draws a PolygonalPath.
        PathDrawing.drawPolygonalPath(g, current_transform, approx_path);
    }

Interaction mode. We will list the possible interaction modes using an Enum. An enum type is just a finite list of named "states" such as our possible interaction interaction modes.

    public enum InteractionMode {
        DO_NOTHING, ZOOM_IN, ZOOM_OUT, MOVE_POINT, DELETE_POINT, ADD_POINT;
    }

    private InteractionMode current_mode = InteractionMode.DO_NOTHING;

So, our interaction mode is in one of six possible states. We keep track of our current mode of interaction, and set it by default to DO_NOTHING.

The following method allows the interaction mode to be changed (by the menu in our case). When the mode is changed, we update the status bar with directions for the user.

    public void setInteractionMode(InteractionMode new_mode) {
        current_mode = new_mode;
        switch (current_mode) {
            case DO_NOTHING:
                postStatus("Welcome to the Fractal Path Explorer.");
                break;
            case ZOOM_IN:
                postStatus("Click to zoom in to a point.");
                break;
            case ZOOM_OUT:
                postStatus("Click to zoom out from a point.");
                break;
            case MOVE_POINT:
                postStatus("Drag a point to change the path.");
                dragging_point = false;
                break;
            case DELETE_POINT:
                postStatus("Click a point to delete.");
                break;
            case ADD_POINT:
                postStatus("Click a point to add to the path.");
                break;
        }
        repaint(); // we display different stuff in diffent modes.
    }

Oracle's tutorial on Enum Types describes this use of the switch statement.

More drawing. We again use the switch statement when finishing our drawing routine. Here is an outline of the routine:

    public void paintComponent(Graphics gfx) {
        super.paintComponent(gfx);
        Graphics2D g = (Graphics2D) gfx;
	// ... omiting stuff ...
        PathDrawing.drawPolygonalPath(g, current_transform, approx_path);

	// display stuff based on mode
        switch (current_mode) {
            case MOVE_POINT:
                drawGamma(g); // Draw the curve gamma and its vertices.
                break;
            case DELETE_POINT:
                drawGamma(g);
                break;
            case ADD_POINT:
                // Draw the curve gamma...
		// Draw the midpoints of the edges of gamma...
		break;
        }
    }

    /** Draw the curve gamma and its vertices. */
    private void drawGamma(Graphics2D g) { ... }

In the MOVE_POINT and DELETE_POINT cases, we display little disks around the vertices of gamma so that the user can click the vertices.

In the ADD_POINT case, we draw the midpoints of edges, so the user can click a midpoint to add it to the vertex list.

Mouse interaction. We use a similar method to handle a user's mouse manipulations. The following explains how we handle mouse clicks:

    public void mouseClicked(MouseEvent me) {
        Complex z = this.toMathCoordinates(me.getPoint());
        switch (current_mode) {
            case ZOOM_IN:
                scale(0.8, z);
                break;
            case ZOOM_OUT:
                scale(1.25, z);
                break;
            case DELETE_POINT: // See if a vertex was clicked, and if it was delete it.
                EdgeIterator it = gamma.iterator();
                if (it.hasNext()) {
                    it.next(); // skip first edge.
                }
                int vertex = 1;
                while (it.hasNext()) {
                    // Get the starting point of the segment, and convert it to 
                    // screen coordinates:
                    Point2D pt = this.toScreenCoordinates(it.next().startingPoint());
                    if (pt.distance(me.getPoint()) < 5) {
                        // We enter here if the screen distance from the click to the 
                        // vertex is les than 5 pixels.
                        setGamma(gamma.removePoint(vertex));
                        break; // exit the while statement
                    }
                    vertex = vertex + 1;
                }
                break;
            case ADD_POINT:
		// Check if a midpoint was clicked. If it was, we add the point to gamma...
                break;
        }
    }

Summary. That explains the main ideas used to build the FractalPathPanel class.

I tried to present the class in a similar way to how the code should be written. Namely, you write some basic methods first, and test them. Then you add a few more methods which do something new and test it. Then you repeat until you have built the whole program. By thinking of the class in terms of the different things it does, you reduce a long programming exercise into simpler more accessible problems.

To see the source code of this class, see FractalPathPanel.java.

Interacting with graphics

A simple interactive program

Fractal Path Manipulator

The Window.

The Panel.

Related Links: