Goals
- Build, test and document a vehicle that wins the 2005 DARPA Grand Challenge
- Qualify for the race by successfully completing the NQE
- Perform publishable research on state-of-the-art problems in robotics
- Effectively manage the "signal to noise" ratio of a large project
|
Spiral 4.1 Goals (W4-W6):
- Develop tools required for efficient operations over the summer
- Analyze feasibility of system specs and revise as needed - Who is working on this?
- RMM: the team coordinators will be responsible for this (through whatever mechanism they choose). The System Specification page has been updated to allow explicit signoff. We will present the system spec at the PDR.
- ADS: The system spec (re-reading) seems to be really rather light on details, this is necessary given the complexity of the project and the team structure, but I am concerned that the detailed specifications at the module level are not clearly defined, we have some broad ones, but for the newer modules in particular I don't think we have clear definitions of what they need to do, to what accuracy and at what speed. This is something that I will be working on as part of SuperCon to some extent, however it seems that it would be really useful to us before designing how a module works, to start by designing it in black-box form, i.e. design in detail it's interfaces what goes in/out and build the shell of the module that just handles the in/out communications *first*.
- Update SURF project plans (including new capabilities) - What does this mean/entail over and above the decisions that were made in the Friday Planning meeting before the holiday?
- RMM: We didn't actually work out the SURF projects for everyone; just the teams that they would be part of. Team coordinators + RMM are responsible for doing this. Will be documented thorugh SURF progress reports due on 18 Jul
- ADS: ok - I currently have some concerns about this for W4, as there are some define/design bugs which need to be cleared up in this period, and I am not sure that we have defined the projects clearly enough for people to take them and start work on them and produce what we wanted - i.e. I think there are several ones that are loosely enough defined for (particularly new team members) to get the wrong idea about what the project's aims are, one example is the DFE/DBS/Safety Factor project that I have discussed with Lars, although perhaps a more critical one is simulation, to my knowledge we don't have details of what specifications we want from a simulation suite - other than 'as good as possible' which works up to a point, but I think simulation is an area where we may be better off (in terms of resource allocation) working more on determining the limitations of our models than developing new ones.
- Return Alice to running condition, including final sensor mounts
- 50 miles of desert driving on moderate roads (ala Stoddard Wells) at up to 20 mph (with stops), using new debugging/logging tools - not happy about this distance spec, why is the distance important at this point? - module tool development is likely to not need long distances to verify performance
- RMM: Agree that 50 miles may not be needed. We should discuss and decide on 5 Jul
|
Spiral 4.2 Goals (W7-W10):
- Complete design & implementation of race vehicle; update specs
- Decide on any
new final capabilities and components that will be used in the race
- 175 miles representative of the expected course in 10 hours (not including pauses), w/ manual reset after failure
- Reassign components and ralign teams, as needed
|
|
Spiral 4.3 Goals (W11-W13):
- Completion of 175 miles in 8 hours or less, w/ automatic recovery from failures
- For each terrain type in System Specification, drive 5X max mileage at average listed speed
-
Demonstrate and commit to new capabilities
- Document research results and
submit outline potential papers
|
Spiral 4.4 Goals (W14-W16):
[Spirals | Test Plan | Edit]
-
Completion of 175 miles in 8 hours or less, w/ automatic recovery from failures
-
For each terrain type in Summer 2005 Test Matrix, drive 5X max mileage at average listed speed
- Complete implementation of all race-critical modules (after pruning)
- Optimize hardware and software for desert racing, consistent with System Specification
- RMM, 4 Sep: Drop specification for operating with 1 computer failure; simplifies design
- Test and demonstrate ability to handle all obstacles to be encounted in NQE
- Optimize race operations, including checklists, log analysis, rapid repair
- Accumulate as much driving as possible, aimed at finding locations of potential failure but avoiding irreperable damage to Alice
- Verify operation in special terrain types (e.g. washouts, ruts,
water, underpasses)
- Multiple test managers per week (lead in bold), coordinating to track/manage/drive activities
</table>
|
Objectives
Spiral 4.1
Summer W4 Critical
- (All) Assign all team-level Spiral 4.1 objectives to individuals or subteams
- (V/E) Analysis of replacement time for all hardware; spare parts ordered
- (V/E) Decide on whether or not to use serial or SDS [bug #1907] - Definitely want to go with ruggedised solution
- RMM: Up to V/E to decide; don't plan to discuss at IPT
- (Pln) Define all supercon interfaces (to planning, terrain, veh/emb), and verify all other planning interfaces
- (Pln) Finalize logging/playback unified interface (timber through gui and command line) - I think that playback means too many different things to different people, this needs to be clearly defined
- RMM: True. Can you you suggest a more specific wording. Planning will be responsible for doing this => don't plan to discuss at IPT
- ADS: I would build on the split-at-map idea for playback, as I don't think that it is necessary to replay the entire system in closed loop. As I see it there are three key forms of playback:
- Playing back recorded actions of the system, (i.e. the pre-recorded inputs and outputs for each module) and analysing the inputs and outputs for applicable modules to verify that their decisions/outputs which were produced at some pre-defined instant in the past were reasonable - *note that this requires the ability to fast-forward/rewind at speeds > real-life*
- Terrain - Ability to playing back raw sensor-outputs (i.e. what the LADARs etc output) *at real-life speeds* into the feeder modules which would actually be running (in real time) and processing the raw sensor outputs and creating 'new' elevation map outputs which would then be fed into FusionMapper (again all real time) - this will allow Terrain to use real sensor data when developing
- Planning - ability to 'play' *in real-time* a stream of map-deltas that were created either from a static map (note that the map-deltas would need to be sent at a speed proportional to how quickly Jennifer is moving at the time) that was manully created, OR playback the map-deltas that were output by FusionMapper from 'real' (i.e. Alice driving around) sensor data - although this is more accurate, it is also less flexible as we can't test any map scenario we want
- ADS (cont'd): I think that (1) could be realised using SkynetMessages - although I think that a better way to do this would be to have the GUI log the information necessary to playback (with a speed control for the playback the GUI) the actions of Alice - as we already log the inputs and outputs of each module (as necessary) with Timber/our own log files we probably already have the information that would be useful for analysing problems in a particular module. And with the GUI logging we could easily find what the interesting point(s) were, and a lot of information is displayed on the GUI now (actuator desired & actual), planner stages, and current map. Plus doing it this way would make it **MUCH** easier as we don't have to mess around with the timing of modules at all - the GUI is just scanning backwards and forwards through a log file of all data it displays, no other modules are involved. We would also need a timestamp displayed in the GUI so that once the interesting section has been identified you could find the specific module ins/outs as desired.
- ADS (cont'd): (2) and (3) should be easier as they are only required to run in real-time, and hence there are no timing issues to consider, we just have to work on two already identified issues - (a) the ability to record and then playback in real-time the raw-sensor outputs - this should not be too hard given the format the data is in, although it would need state recorded as well, I still think that this should not be difficult to implement. (b) the ability to have FusionMapper break up a static map and send it, or replay it's map-deltas at a speed dependent upon the current speed in the simulator, the breaking up of a static map might be interesting, but I don't think that it should be too hard, if we only sent 1 delta/cell this might not be super realistic, but I don't think that this is a big issue compared to the other known simulation issues. And replaying FusionMappers sent map-deltas should in principle be very easy - adjust the speed dependent upon the vehicle speed should also be relatively easy
- ADS (cont'd): I've been playing with these ideas for a while now and so far this is my best iteration, and I think that it's the easiest, fastest and clearest way to do Playback in all it's glory, and will build well on the tools that we already have and reduce significantly the total work-load required for implmentation - but if anyone has better ideas please let me know! :)
(Ter) Verify high speed LADAR operation (from W1)
(Ter) Define and document final sensor configuration
(Ter) Define and document final module configuration
(Ter) Investigate and decide if sim env is useful - This is a big one, Gazebo development needs to have a deadline, beyond which it is cut and resources are re-allocated
- RMM: Deadline is Summer PDR
- ADS: I think that this could be too late, I remember that we were saying if Gazebo would take 10 hours then it was worth it, if it was 120 it wasn't, I think that the deadline should be a two stage one, with the stages being at the ends of W4 and W5, and that we could define explicit W4 and W5 objectives for Gazebo based on the outstanding bugs. I have a real concern that we have a reasonably large group working on simulation, and that the learning curve for developing Gazebo is steep, and I at least (and I know JG & others share my sentiment) have deep reservations about Gazebo's usefulness. It seems that by committing resources to Gazebo until PDR we are working on a simualtion environment whose eventual use/performance is questionable, as resources are tight and we need reliable and effective simualtions ASAP I think that we would make more progress working on playback and enhancing asim (better actuator dynamics etc) as it is in real C++, and does not require specialised hardware - for me this last one is a *big* plus as looking forward a lot of people will be developing on Jennifer, and I am wary of us using two machines exlusively per each instance of Gazebo, and that especially with 20+ people in the shop ssh-ing, this will put too heavy a load on the remainging machines - there are some things that we can do about this with the laptops - namely increasing their RAM, but we have not yet been able to test what impact this would make. I know that Gazebo would allow us to test in 'closed-loop', but I think that this is not a significant issue, we are not intending to have the planner consider the % of unblocked Field-of-View for the terrain sensors when considering where to drive, and provided that we can plan off replayed maps of real data, I see no reason why gazebo is necessary given its known limitations.
(IPT) Decide on ownership of overlapping modules
- process management (V/E vs pln) - thought that this had been resolved as going to planning (like skynet) as Lusann/Henry are dealing with this
- RMM: Agree. IPT (V/E and Pln) just needs to sign off
- learning (pln vs ter) - probably requires moving resources as well
- RMM: any thoughts about who/what? Pln has 12 people, Ter has 8
- gimbal pointing (pln vs ter vs V/E) - thought that this was going to terrain, but that they will listen to the trajs sent to TrajFollower
- RMM: Agree. IPT (V/E, Pln Ter) just needs to sign off
Summer W4 Baseline
- (V/E) Complete OBD II installation/testing - still want to know what info will come from OBD2 and where it will go
- RMM: same as last year, I think - OBD II will feed into adrive and astate. Will tell us whether we are rolling (for astate) and will feed info to SuperCon for fault management
- (V/E) Convert sparrow to GTK - don't think that this is worth it
- RMM: I'm not sure who is driving this change. We should discuss with Lars. I'll take the action to do that [bug #1952]
- (V/E) First cut mounting of all available sensors: Reigl, new SICKs, cameras
- (V/E) Create wiki page to document methods to meet 12-hour catastrophic recovery spec
- (Pln) Complete all
non-interface-related planning define/design/decide objectives
- (Pln, Ter) Determine what will and will not be covered by SuperCon
- (Ter) Sensor drivers cleaned up and interfaces defined - some of this will be done in W4
- (Ter) Determine precise cause of catastrophic astate failures
- (IPT) Decide on progress tracking metrics
|
Summer W5 Critical
- (V/E) AC functional
- (V/E) New small hardware installed (e.g. brake cylinders, securing wires, etc.)
- (V/E) Steering driver rewritten
- (V/E) All sensors mounted in first-cut mounts
- (Pln) Finalize logging/playback of raw data and skynet messages (implementation and documentation for one module)
- (Pln) Finish GUI API
- (Ter) Efficient offline calibration procedure ready and tested in the field
- (Ter) Sensor drivers fully implemented and done
- (Ter) Astate catastrophic failures fixed
Summer W5 Baseline
- (V/E, ops) Checklists completed and tested close to home (field tests, autonomous ops, emergency procedures)
- (Pln) Identify likely "failure" conditions and plans to address them
- (Pln) Complete basic algorithmic rough pseudo-codish design of planning software - too vague
- RMM: the red color is supposed to signify items that are too vagues, so I agree. I'm not even sure that I understand what this one means. Do you?
- ADS: I think that the idea was to have black-box designs for the 'outside' of all modules and some form of pseudo-code for the inside of modules, I am a big fan of having complete black-box designs for all modules at this point though - perhaps even having the shells of the modules (i.e. just the interfaces) written and compiling as well, if we could do this I think we would be in very good shape, and it might also be an idea to do this 'together' - i.e. as many modules as possible, as then we could coordinate it with some lectures on how to set-up skynet messages in/out of a module, sparrow displays and some other good module tips
- (Ter) Astate small jumps (~1-10m) fixed - does this mean new KF? - is there time?
- RMM: up to terrain to decide how. I think what we want to do is decide when. Summer PDR?
- ADS: I am tempted to say W5 as the provisional goal, as astate jumps are very detrimental to testing generally, and after W5 we will hopefully be testing much more frequently
- (Ter) MapTalker implemented using TCP/IP
- (All) Assess feasibility of System Specification (range, obstacles, fault toleraance, variable safety factor, rapid repair) and recommend changes, if needed
- (All) Code review of critical software: hardware drivers, adrive, astate, skynet, TrajFollower
- (Doc) Documentation templates for component and interface pages defined in place for PDR
- (IPT) Define interfaces for all components in status charts
|
Summer W6 Critical
- (V/E) Demonstrate monitoring of autonomy level - what does this mean?
- RMM: I believe this means that adrive should monitor and log whether we are in fully autonomous operations, partial (steering, throttle, brake), or manual operations. Agree we should make more specific
- (Pln) Finish instant replay ability, including documentation
- (Pln) Characterize performance envelope for average speeds on spec'd terrains: max steering angle/rate versus speed, max decel w/out splipping, max roll angle - how? - our data (Jeff) or from STI?
- RMM: good question. Planning should figure this out in W4. We need this in order to make sure that our System Specification is feasible (part of Spiral 4.1 goals)
- (Ter) Take data for spec'd terrains where Alice can't yet drive autonomously at top spec'd speed
- (Ter) Dust/mud characterization
- (Tst) Demonstrate reliable operation of critical components (cooling, power, wiring, SDS, LADARs, stereo, state estimation, OBD II) - not sure how this will be tested - need to define more tightly
- RMM: I agree. Suggestions?
- ADS: If I was being sadistic I would say that we leave Alice in the sun with everything running in the desert for a day, perhaps with heaters inside as well and monitor the temperature inside the computer case (prefereably electronically record the data and then analyse). State estimation could be tested by driving Alice (manually) in a straight/curvy line at el-mirage and having a small app. to add noise to the GPS signal as per bad-GPS regions (or completely block GPS data) and monitor how astate performs, and for what distance - S-bends would be evil :). Power - I think we should just set Alice up and run her until the batteries run out and determine how long this actually is with everything running, we could repeat the experiment with Alice's engine on and high-idling to simulate driving maybe - or as part of the later test we could just have her circle stoddard wells for a day - continuously autonomously until she ran out of diesel. Wiring - take a spring balance and tug the wire with a measured force (defined in advance) and check the connections before/afterwards
- (All) GUI interface, logging and playback implemented in all modules (including "instant replay" through gui) for debugging autonomous runs and offline testing with stored data
Summer W6 Baseline
- (Race Teams) Complete all spiral 4.1 objectives from team GOTChAs: Project, Veh/Emb, Planning, Terrain, IPT
- (Bld) Build process defined and documented (stable branch, test branch, merge process); validate in W6
- (Mod) Modeling environment defined and available for on-campus testing during W6 - There aren't enough develop modelling suite bugs/actions for this to be realistic at the moment - need to detail development of asim & playback with the aim of producing a split simulation
- RMM: the modeling team will need to pull together a plan for this and implement by W6. Their call about whether we do a split simulation, but it seems likely
- ADS: see earlier comments about this and possible resolution
- (Ops) Checklists updated for W6 field testing
- (Sys) Resolve all SysAdmin blockers from Bugzilla
- (All) Update (SURF) project plans, linked to project objectives or possible new technologies, and present go/no-go criteria + implementation plans
</table>
Spiral 4.2
- Final versions of race hardware installed - Veh/Emb
- Microcontroller for actuation, sensing, w/ spares
- Gimballed sensor platform installed and interfaces operational
- Fail-safe e-stop system, using R/C + DARPA hardware
- Ignition and transition actuation systems, including SW interface
<li> Sensor integration complete - Planning, Terrain
- All sensors feeding into cost map
- Sensors detecting obstacles within specified error margins
- Sensors capable of determining sensor health; errors detected by supervisory control
<li> Detection of failures and ability to operate (after manual reconfig) with 2 sensor + 1 computer failure - Planning, Terrain
<li> High speed, autonomous driving (at least average speed from system spec) - Planning, Terrain
- Planning using fused cost map
- Online sensor recalibration
|
- Automatic e-stop and recovery from CPU failures
- System performance characterization
</table>
- Completion of 175 miles in 10 hours or less elapsed time by W9 field test - All
- Course to be selected independent of test team and provided 2 hours prior to start
- Sections of the course to be isolated to allow high speed driving
- Final sensor suite (all sensors installed and operating)
- Fault detection and isolation after manual restart
<li> Submit technical paper to DARPA (15 Aug) - All
- Extended abstracts for all SURF projects available on the web
- All components documented on the wiki
<li> Average of 3 days/week of desert testing, with average of 6 hours/day autonomous driving by W9 - IPT
<li> Proof of concept implementation of new technologies w/ go/no-go decision for race - All + IPT
- No new technology to be introducted after CDR
</ol>
Gate: Critical Design Review (CDR), 17 Aug
Spiral 4.3
- Demonstration of NQE capable vehicle
- Include ability to avoid man-made obstacles on course
- All components and interafaces documented on the wiki
- 1000 miles of autonomous driving (W10-W12)
- Analyze logs for all failures and implement improvements w/in 1 week
- Demonstrate ability to handle dead end paths, including stuck on unknown obstacle (eg, barbed wire fence)
- Ability to detect, isolate and recover from up to 2 sensor + 1 computer failures
- Completion of 175 miles in 8 hours or less elapsed time
- All technologies that will be used in the NQE, race operational
- Fault detection, isolation and recovery
Race and beyond
- Continuous improvement of vehicle performance
- Submission of 5 or more papers to ICRA
Technical Challenges
- Robust vehicle capable of surving testing without damage
- Spring term objectives not fully realized; we are starting from behind
- Very aggressive schedule between beginning of summer, PDR, CDR and race
- Many untested technologies under consideration; could serve as a distraction
- High level decision making and learning not yet planned
- High speed technical driving in rough terrain not yet demonstrated; likely to be hard and risky (to Alice)
- System still susceptable to component and interface failures, resulting in substantial downtime
- Large team, with diverse background, skills, experience, and availability
|
Approach
- Rely on team structure and management tools refined during academic year
- Detailed set of objectives through the summer, to insure we are identifying and solving problems early and keeping the entire team working on appropriate tasks
- Test-focused timeline and test plan, with weekly desert testing
- Min two days/week of shop time for Alice
- Project, team meetings consilidated into single day
- Focus on blockers day before tests; development of new technologies via simulation during tests
- Maintain visible indicators of progress against project goals
- Tracking of open bugs, weighted by severity
- Tracking of system-level metrics, in highly visible form
</table> |
|
|
|
|