Mar 22, 2020

Real-Time 3D with Python Part IV: Cityscape

In the introduction I explained the idea: Using Python to program old skool demo style real-time vector graphics. In Part I I set up a simple 3D wireframe object, rotated it, and drew it on the screen, and in Part II added surfaces and perspective. Part III was about calculating object surface visibility and shading. This part sets up an XML reader to define the objects and other properties using a common file format (XML), and adds the code needed to work with multiple objects.

The program code and XML files for all parts can be downloaded from github. Note that the full code is not found below - to run this, use github code.

It's a World Out There


Previously, I only had a single object shown in the middle of the screen. Now I am going to add many objects, so that they all have their own coordinates. In addition, they can have their own rotation defined, so I may need multiple "angle sets". What I will end up with all the changes is this:


To get there, let's add two new classes, VectorAngles and VectorPosition:

class VectorAngles:

    """
    Angles for rotating vector objects. For efficiency, one set of angles can be used for many objects.
    Angles are defined for axes X (horizontal), Y (vertical), Z ("distance") in degrees (360).
    
    @author: kalle
    """
    def __init__(self):
        self.angles = np.array([0.0, 0.0, 0.0])
        self.angleScale = (2.0 * np.pi) / 360.0 # to scale degrees.
        self.angName = ""
        self.rotationMatrix = np.zeros((3,3))
        self.rotateAngles = np.array([0.0, 0.0, 0.0])
        self.rotate = np.array([0.0, 1.0, 0.0])
    
    def setAngles(self, angles):
        # Set rotation angles to fixed values. 
        self.angles = angles
     
    def setRotateAngles(self):
        self.rotateAngles += self.rotate
        for i in range(3):
            if self.rotateAngles[i] >= 360: self.rotateAngles[i] -= 360
            if self.rotateAngles[i] < 0: self.rotateAngles[i] += 360

    def setRotationMatrix(self):
        # Set matrix for rotation using angles.
        
        (sx, sy, sz) = np.sin((self.angles + self.rotateAngles) * self.angleScale)
        (cx, cy, cz) = np.cos((self.angles + self.rotateAngles) * self.angleScale)
 
        # build a matrix for X, Y, Z rotation (in that order, see Wikipedia: Euler angles) including position shift. 
        # add a column of zeros for later position use
        self.rotationMatrix = np.array([[cy * cz               , -cy * sz              , sy      ],
                                        [cx * sz + cz * sx * sy, cx * cz - sx * sy * sz, -cy * sx],
                                        [sx * sz - cx * cz * sy, cz * sx + cx * sy * sz, cx * cy ]])
    
class VectorPosition:

    """
    A vector object defining the positions of other objects in its nodes (see VectorObject).
    
    @author: kalle
    """
    def __init__(self):
        self.position = np.array([0.0, 0.0, 0.0, 1.0])
        self.angles = VectorAngles()
        self.nodes = np.zeros((0, 4))                   # nodes will have unrotated X,Y,Z coordinates plus a column of ones for position handling
        self.rotatedNodes = np.zeros((0, 3))            # rotatedNodes will have X,Y,Z coordinates
        self.objects = []                               # connects each node to a respective VectorObject
        self.objName = ""
        
    def addNodes(self, node_array):
        # add nodes (all at once); add a column of ones for using position in transform
        self.nodes = np.hstack((node_array, np.ones((len(node_array), 1))))
        self.rotatedNodes = node_array # initialize with nodes
        
    def addObjects(self, object_list):
        self.objects = object_list

    def rotate(self):
        # apply a rotation defined by a given rotation matrix.
        matrix = np.vstack((self.angles.rotationMatrix, np.zeros((1, 3))))
        # apply rotation and position matrix to nodes
        self.rotatedNodes = np.dot(self.nodes, matrix) + self.position[0:3]

As you may note, I have just moved the code in VectorAngles from VectorObject, where it still was in Part III. Now that we have multiple objects, most of them share their rotation (angles), so it is better to process that part just once per rotation set.

VectorPosition is new but very simple. The nodes hold the original coordinates of each object, and rotatedNodes hold the coordinates after rotation according to angles. In addition, there's a list objects to maintain the connection between the coordinates and the respective object. In a way, VectorPosition is very similar to a VectorObject, but it does not have surfaces. Instead, each of its nodes just represents the position of a VectorObject.

Now, when rotating the objects in VectorViewer, rotation is done in two steps: first the positions above are rotated and copied into the actual objects, which are then rotated in the second step:

    def rotate(self):
        """ 
        Rotate all objects. First calculate rotation matrix.
        Then apply the relevant rotation matrix with object position to each VectorObject.
        """
                
        # calculate rotation matrices for all angle sets
        for VectorAngles in self.VectorAnglesList:
            VectorAngles.setRotateAngles()
            VectorAngles.setRotationMatrix()
        
        # rotate object positions, copy those to objects.
        self.VectorPos.rotate() 
        for (node_num, VectorObj) in self.VectorPos.objects:
            VectorObj.setPosition(self.VectorPos.rotatedNodes[node_num, :])

        # rotate and flatten (transform) objects
        for VectorObj in self.VectorObjs:
            VectorObj.updateVisiblePos(self.objMinZ) # test for object position Z
            if VectorObj.visible == 1:
                VectorObj.rotate() # rotates objects in 3D
                VectorObj.updateVisibleNodes(self.objMinZ) # test for object minimum Z
                if VectorObj.visible == 1:
                    VectorObj.transform(self.midScreen, self.zScale) # flattens to 2D, crops X,Y
                    VectorObj.updateVisibleTrans(self.midScreen) # test for outside of screen

Note how first all possible "angle sets" are processed so that the rotation matrices are ready for use. Also, when rotating the objects, I am updating a visible property; now that we have many objects moving around, it may be possible to stop processing some of them, if we know they will not be visible anyway. First, I am testing for object position Z coordinate. In my world, the viewer is sitting at the origo, so anything with a negative Z coordinate is behind her. I have actually defined an objMinZ property to not draw anything which is too close. So, if the object is indeed too close, judging by its position, it will not be shown and thus also rotating it will be unnecessary.

A second test, for the objects surviving the first, is to check is the object is on screen at all. This is done by checking the minimum and maximum transNodes i.e. X and Y coordinates against the screen boundaries. (For the actual code for these simple checks, see the github files.) 

City Planning


To set up the objects, I created an XML file. In the code, I imported some new modules to do that:

import os
from operator import itemgetter
import copy
import xml.etree.ElementTree as et

And then, before launching the VectorViewer, the part below reads the XML and adds the objects. (It's a lengthy part so this is just the beginning.) If you run the github code, you will need to change the data directory to point to the right place. Handling XML is very easy; the ElementTree.parse builds a structure which can then be iterated or searched for the data as required. Note that the code I built contains almost no error checking whatsoever for missing or incorrect data.

if __name__ == '__main__':
    """ 
    Prepare screen, read objects etc. from file.
    """

    # set data directory
    os.chdir("D:\kalle\Documents\Python")

    # set screen size
    # first check available full screen modes
    pygame.display.init()
    # disp_modes = pygame.display.list_modes(0, pygame.FULLSCREEN | pygame.DOUBLEBUF | pygame.HWSURFACE)
    # disp_size = disp_modes[4] # selecting display size from available list. Assuming the 5th element is nice...
    disp_size = (1280, 800)

    vv = VectorViewer(disp_size[0], disp_size[1])

    # read data file defining angles, movements and objects
    vecdata = et.parse("vectordata cityscape.xml")

    root = vecdata.getroot()
    
    for angles in root.iter('vectorangles'):
        ang = VectorAngles()
        ang.angName = angles.get('name')
        ang.angles[0] = float(angles.findtext("angleX", default="0"))
        ang.angles[1] = float(angles.findtext("angleY", default="0"))
        ang.angles[2] = float(angles.findtext("angleZ", default="0"))
        vv.addVectorAnglesList(ang)
   
    for vecobjs in root.iter('vectorobject'):
        vobj = VectorObject()
        vobj.objName = vecobjs.get('name')
            
    ...

The XML code is also rather lengthy and below is the beginning, defining a lightsource, one set of angles, and one vector object.

<?xml version="1.0"?>
<vectordata>
 <lightsource>
  <!-- set lightsource position for shading. If any object has the "lightsource" property, these will be overridden. -->
  <lightposition>
   <lightpositionX>600</lightpositionX>
   <lightpositionY>200</lightpositionY>
   <lightpositionZ>800</lightpositionZ>
  </lightposition>
 </lightsource>
 <vectorangleslist>
  <!-- set up angle sets for rotations. Values given are starting values. -->
  <vectorangles name="viewer">
   <angleX>0</angleX>
   <angleY>90</angleY>
   <angleZ>0</angleZ>
  </vectorangles>
 </vectorangleslist>
 <vectorobjectslist>
  <!-- set up vectorobjects. Each object must have a position, a reference to angles used, a list of nodes, and a list of surfaces. -->
  <!-- minshade (must be between 0 and 1) determines "shade multiplier" when surface is at 90 degree angle to light source. Use 1.0 to keep full color value ie. no shading. -->
  <!-- use "copyfrom" property (e.g. <vectorobject name="cube2" copyfrom="cube1">) to copy objects; properties like position and colors can then be re-defined, and initangles specified. -->
  <!-- object can have default values for surface color, edgewidth, showback (e.g. "defcolor"). These can be overridden for each surface. --> 
  <vectorobject name="house1_1">
   <!-- house1: a small house with gabled roof -->
   <position>
    <positionX>-120</positionX>
    <positionY>0</positionY>
    <positionZ>200</positionZ>
   </position>
   <anglesref>viewer</anglesref>
   <minshade>0.2</minshade>
   <defcolor>
    <defcolorR>255</defcolorR>
    <defcolorG>255</defcolorG>
    <defcolorB>255</defcolorB>
   </defcolor>
   <defedgewidth>0</defedgewidth>
   <defshowback>0</defshowback>
   <nodelist numnodes="10">
    <node ID="0">
     <nodeX>50</nodeX>
      <nodeY>0</nodeY>
      <nodeZ>-100</nodeZ>
    </node>
    <node ID="1">
     <nodeX>50</nodeX>
      <nodeY>0</nodeY>
      <nodeZ>100</nodeZ>
    </node>
    <node ID="2">
     <nodeX>-50</nodeX>
      <nodeY>0</nodeY>
      <nodeZ>100</nodeZ>
    </node>
    <node ID="3">
     <nodeX>-50</nodeX>
      <nodeY>0</nodeY>
      <nodeZ>-100</nodeZ>
    </node>
    <node ID="4">
     <nodeX>50</nodeX>
      <nodeY>100</nodeY>
      <nodeZ>-100</nodeZ>
    </node>
    <node ID="5">
     <nodeX>50</nodeX>
      <nodeY>100</nodeY>
      <nodeZ>100</nodeZ>
    </node>
    <node ID="6">
     <nodeX>-50</nodeX>
      <nodeY>100</nodeY>
      <nodeZ>100</nodeZ>
    </node>
    <node ID="7">
     <nodeX>-50</nodeX>
      <nodeY>100</nodeY>
      <nodeZ>-100</nodeZ>
    </node>
    <node ID="8">
     <nodeX>0</nodeX>
      <nodeY>125</nodeY>
      <nodeZ>-75</nodeZ>
    </node>
    <node ID="9">
     <nodeX>0</nodeX>
      <nodeY>125</nodeY>
      <nodeZ>75</nodeZ>
    </node>
   </nodelist>
   <surfacelist>
    <surface ID="0">
     <nodelist>
      <node order="0" refID="0"/>
      <node order="1" refID="1"/>
      <node order="2" refID="5"/>
      <node order="3" refID="4"/>
     </nodelist>
    </surface>
    <surface ID="0">
     <nodelist>
      <node order="0" refID="1"/>
      <node order="1" refID="2"/>
      <node order="2" refID="6"/>
      <node order="3" refID="5"/>
     </nodelist>
    </surface>
    <surface ID="0">
     <nodelist>
      <node order="0" refID="2"/>
      <node order="1" refID="3"/>
      <node order="2" refID="7"/>
      <node order="3" refID="6"/>
     </nodelist>
    </surface>
    <surface ID="0">
     <nodelist>
      <node order="0" refID="3"/>
      <node order="1" refID="0"/>
      <node order="2" refID="4"/>
      <node order="3" refID="7"/>
     </nodelist>
    </surface>
    <surface ID="0">
     <nodelist>
      <node order="0" refID="4"/>
      <node order="1" refID="5"/>
      <node order="2" refID="9"/>
      <node order="3" refID="8"/>
     </nodelist>
    </surface>
    <surface ID="0">
     <nodelist>
      <node order="0" refID="5"/>
      <node order="1" refID="6"/>
      <node order="2" refID="9"/>
     </nodelist>
    </surface>
    <surface ID="0">
     <nodelist>
      <node order="0" refID="6"/>
      <node order="1" refID="7"/>
      <node order="2" refID="8"/>
      <node order="3" refID="9"/>
     </nodelist>
    </surface>
    <surface ID="0">
     <nodelist>
      <node order="0" refID="7"/>
      <node order="1" refID="4"/>
      <node order="2" refID="8"/>
     </nodelist>
    </surface>
   </surfacelist>
  </vectorobject>

The object above is a simple house with a gabled roof. We need ten nodes; four for the corners on the ground, four for the corners at the top of the walls, and two for the gabled roof. Furthermore, there are eight surfaces; four for the walls, and four for the roof. The color of each surface can be defined individually, but I have only defined a "defcolor", which is then the default.

The position of the object is its position in the world, and its nodes are defined relative to that being the center of the object. The "anglesref" links the object to an angle set of the same name for rotation.

There is also a copying functionality implemented. By setting e.g.
  <vectorobject name="house2_2" copyfrom="house2_1">
   <position>
    <positionX>-700</positionX>
    <positionY>0</positionY>
    <positionZ>-100</positionZ>
   </position>
  </vectorobject>

the object house2_2 will be a copy of house2_1, but its position will be set separately. It is also possible to override some other properties like surface colors, or to rotate the copied object in place before applying it. This makes re-using object data easier. 


Efficient Programming


There must be tons of books, university courses etc. on how to achieve efficiency in programming. I have not read or attended any of them. Back in the 1980's I taught myself programming first in BASIC (I think Beginners' All-purpose Symbolic Instruction Code says it all) by reading instruction manuals and magazines, and by trial and error; then later Assembly by the same method and the Amiga Hardware Reference Manual and Amiga Machine Language (still available at Amazon!).

To me, there must be at least three definitions of efficiency in programming:
  1. how efficiently the program code runs - how quickly it gets the job done.
  2. how efficient it is to program - how long a time it takes for the programmer to create the program for the specified purpose.
  3. how efficient it is to document and maintain the program - how to keep the program running and add changes when requirements change etc.
Arguably, Assembly language excels in (1) but is miserable in (2) and (3), especially if the programmer is a 1980's self-taught demo coder. (Note that while we demo coders seldom did so, there were libraries etc. we could have used even then and in Assembly.) Python, for example, is probably the opposite - not very efficient code in comparison, but way ahead in programmability and maintainability.


No comments:

Post a Comment