We present a challenging new dataset for autonomous driving: the Oxford RobotCar Dataset. Over the period of May 2014 to December 2015 we traversed a route through central Oxford twice a week on average using the Oxford RobotCar platform, an autonomous Nissan LEAF. This resulted in over 1000 km of recorded driving with almost 20 million images collected from 6 cameras mounted to the vehicle, along with LIDAR, GPS and INS ground truth. Data was collected in all weather conditions, including heavy rain, night, direct sunlight and snow. Road and building works over the period of a year significantly changed sections of the route from the beginning to the end of data collection. By frequently traversing the same route over the period of a year we enable research investigating long-term localization and mapping for autonomous vehicles in real-world, dynamic urban environments. The full dataset is available for download at: http://robotcar-dataset.robots.ox.ac.uk
Path-velocity decomposition is an intuitive yet powerful approach to addressing the complexity of kinodynamic motion planning. The difficult trajectory planning problem is solved in two separate, simpler steps: first, a path is found in the configuration space that satisfies the geometric constraints (path planning), and second, a time-parameterization of that path satisfying the kinodynamic constraints is found. A fundamental requirement is that the path found in the first step must be time-parameterizable. Most existing works fulfill this requirement by enforcing quasi-static constraints during the path planning step, resulting in an important loss in completeness. We propose a method that enables path-velocity decomposition to discover truly dynamic motions, i.e. motions that are not quasi-statically executable. At the heart of the proposed method is a new algorithm – Admissible Velocity Propagation – which, given a path and an interval of reachable velocities at the beginning of that path, computes exactly and efficiently the interval of all the velocities the system can reach after traversing the path, while respecting the system’s kinodynamic constraints. Combining this algorithm with usual sampling-based planners then gives rise to a family of new trajectory planners that can appropriately handle kinodynamic constraints while retaining the advantages associated with path-velocity decomposition. We demonstrate the efficiency of the proposed method on some difficult kinodynamic planning problems, where, in particular, quasi-static methods are guaranteed to fail.
In unlabeled multi-robot motion planning, several interchangeable robots operate in a common workspace. The goal is to move the robots to a set of target positions such that each position will be occupied by some robot. In this paper, we study this problem for the specific case of unit-square robots moving amidst polygonal obstacles and show that it is PSPACE-hard. We also consider three additional variants of this problem and show that they are all PSPACE-hard as well. To the best of our knowledge, this is the first hardness proof for the unlabeled case. Furthermore, our proofs can be used to show that the labeled variant (where each robot is assigned a specific target position), again, for unit-square robots, is PSPACE-hard as well, which sets another precedent, as previous hardness results require the robots to be of different shapes (or at least in different orientations). Lastly, we settle an open problem regarding the complexity of the well-known Rush-Hour puzzle for unit-square cars in environments with polygonal obstacles.
We present a novel approach to real-time dense visual simultaneous localisation and mapping. Our system is capable of capturing comprehensive dense globally consistent surfel-based maps of room scale environments and beyond explored using an RGB-D camera in an incremental online fashion, without pose graph optimization or any post-processing steps. This is accomplished by using dense frame-to-model camera tracking and windowed surfel-based fusion coupled with frequent model refinement through non-rigid surface deformations. Our approach applies local model-to-model surface loop closure optimizations as often as possible to stay close to the mode of the map distribution, while utilizing global loop closure to recover from arbitrary drift and maintain global consistency. In the spirit of improving map quality as well as tracking accuracy and robustness, we furthermore explore a novel approach to real-time discrete light source detection. This technique is capable of detecting numerous light sources in indoor environments in real-time as a user handheld camera explores the scene. Absolutely no prior information about the scene or number of light sources is required. By making a small set of simple assumptions about the appearance properties of the scene our method can incrementally estimate both the quantity and location of multiple light sources in the environment in an online fashion. Our results demonstrate that our technique functions well in many different environments and lighting configurations. We show that this enables (a) more realistic augmented reality rendering; (b) a richer understanding of the scene beyond pure geometry and; (c) more accurate and robust photometric tracking.
This paper presents quantitative measures of a robot’s physical ability to balance itself actively on a single point, line or area of support. These measures express the ratio of a change in the state of motion of the robot’s center of mass to the amount of action required at the actuated joints in order to produce that change. They therefore represent measures of the gain of the robot mechanism as seen from the point of view of the balance control system. This paper is concerned mainly with ratios of velocities, called velocity gains, and it builds on earlier work by showing how these ratios can be defined and calculated for the case of a general planar or spatial robot balancing on a point, line or general rolling contact, or an area contact with a compliant surface. The paper concludes with three examples of use—design of a triple pendulum, analysis of a hydraulic quadruped, and expressing the physics of planar balancing—followed by a short discussion of gyroscopic balancing.
We propose the concept of a Force-Amplifying N-robot Transport System (Force-ANTS) to coordinate the manipulation forces from a group of robots in order to transport a heavy object in a planar environment. Our approach requires no explicit communication among robots. Instead, we prove that robots can use local measurements of the object’s motion at their attachment points as implicit information for force coordination. A leader (either a robot or human) can guide the whole group towards the destination by applying a relatively small force, whose effect is amplified by the follower robots as they align their forces with the leader’s. Two Force-ANTS implementations are introduced and analyzed, accounting for two different classes of object dynamics: small objects where kinetic friction dominates, and large objects where inertia and viscous friction dominate. Our approach can be used as a modular system for transporting heavy objects of various sizes in many real-life applications. Simulations with up to 1000 robots and experiments using four custom-built robots are conducted to validate our approach. We also conduct human–robot cooperation experiments where the human force is amplified by three follower robots.
This work presents a new control approach to multi-contact balancing for torque-controlled humanoid robots. The controller includes a non-strict task hierarchy, which allows the robot to use a subset of its end effectors for balancing while the remaining ones can be used for interacting with the environment. The controller creates a passive and compliant behavior for regulating the center of mass (CoM) location, hip orientation and the poses of each end effector assigned to the interaction task. This is achieved by applying a suitable wrench (force and torque) at each one of the end effectors used for interaction. The contact wrenches at the balancing end effectors are chosen such that the sum of the balancing and interaction wrenches produce the desired wrench at the CoM. The algorithm requires the solution of an optimization problem, which distributes the CoM wrench to the end effectors taking into account constraints for unilaterality, friction and position of the center of pressure. Furthermore, the feedback controller is combined with a feedforward control in order to improve performance while tracking a predefined trajectory, leading to a control structure similar to a PD+ control. The controller is evaluated in several experiments with the humanoid robot TORO.
We consider the problem of computing a minimum uncertainty path for an autonomous vehicle from a start to a destination location in the presence intermittent sensing, modeled as a stochastic process, in addition to process and measurement noise. We introduce the use of a novel bound on the maximum eigenvalue of the estimation error covariance matrix as the cost function for belief space planning. Our main contributions are three-fold. We first derive an analytic bound on the performance of a state estimator under sensor misdetection (intermittency) occurring stochastically over time. Second, we use this bound as a proxy for the expected maximum eigenvalue evolution in a sample-based path planning algorithm to produce a path that trades off accuracy and robustness. This extends the recent body of work on planning under uncertainty to include the fact that sensors may not provide any measurement owing to misdetection. Computational results demonstrate the benefit of the approach and comparisons are made with the state of the art in path planning in belief space. Third, and finally, we establish theoretically that the proposed algorithm possesses the optimal substructure property, i.e. the algorithm returns an optimal path relative to the bound treated as a proxy for the expected maximum eigenvalue evolution.
This paper focuses on devising a control policy that is inspired by human strategy to enable robots to perform surface–surface contact between a hand-held object (e.g. a box) and the environment (e.g. table). We assume the object’s shape is partially known and consider uncertainties in both the environment and the object grasp. Our analysis on ten untrained subjects indicates that during this task: (1) the subjects start decreasing the angular velocity before the complete alignment (in contrast to existing approaches), and (2) they do not control the contact force to remain at a fixed value. Our study is also consistent with the hypothesis that the subjects determine an over-estimate of the relative angle between the object in hand and the environment. Based on these observations we propose a novel control policy, called Surface-Surface Contact Primitive (SSCP), which can perform the surface–surface alignment task with only partial information about the hand-held object and the environment. Furthermore, SSCP only requires a rough estimate of the surface normal vector and does not rely on either the estimation of contact type (i.e. point or edge contact) or locations of contact points. We evaluate the performance of the proposed controller on a set of robot experiments using two seven-degree-of-freedom robots, one for imposing uncertainty to the environment, and the other to perform the experiment. We show the applicability of the controller on four objects with different geometries, its generalization to four different surface materials, and its robustness to uncertainty in environment and grasp as well as external perturbations.
As service robots become more and more capable of performing useful tasks for us, there is a growing need to teach robots how we expect them to carry out these tasks. However, different users typically have their own preferences, for example with respect to arranging objects on different shelves. As many of these preferences depend on a variety of factors including personal taste, cultural background, or common sense, it is challenging for an expert to pre-program a robot in order to accommodate all potential users. At the same time, it is impractical for robots to constantly query users about how they should perform individual tasks. In this work, we present an approach to learn patterns in user preferences for the task of tidying up objects in containers, e.g. shelves or boxes. Our method builds upon the paradigm of collaborative filtering for making personalized recommendations and relies on data from different users which we gather using crowdsourcing. To deal with novel objects for which we have no data, we propose a method that compliments standard collaborative filtering by leveraging information mined from the Web. When solving a tidy-up task, we first predict pairwise object preferences of the user. Then, we subdivide the objects in containers by modeling a spectral clustering problem. Our solution is easy to update, does not require complex modeling, and improves with the amount of user data. We evaluate our approach using crowdsourcing data from over 1200 users and demonstrate its effectiveness for two tidy-up scenarios. Additionally, we show that a real robot can reliably predict user preferences using our approach.
The paper presents the simulation-based variant of the LQR-tree feedback-motion-planning approach. The algorithm generates a control policy that stabilizes a nonlinear dynamic system from a bounded set of initial conditions to a goal. This policy is represented by a tree of feedback-stabilized trajectories. The algorithm explores the bounded set with random state samples and, where needed, adds new trajectories to the tree using motion planning. Simultaneously, the algorithm approximates the funnel of a trajectory, which is the set of states that can be stabilized to the goal by the trajectory’s feedback policy. Generating a control policy that stabilizes the bounded set to the goal is equivalent to adding trajectories to the tree until their funnels cover the set. In previous work, funnels are approximated with sums-of-squares verification. Here, funnels are approximated by sampling and falsification by simulation, which allows the application to a broader range of systems and a straightforward enforcement of input and state constraints. A theoretical analysis shows that, in the long run, the algorithm tends to improve the coverage of the bounded set as well as the funnel approximations. Focusing on the practical application of the method, a detailed example implementation is given that is used to generate policies for two example systems. Simulation results support the theoretical findings, while experiments demonstrate the algorithm’s state-constraints capability, and applicability to highly-dynamic systems.
This paper presents a demonstration of the trot-to-gallop transition and subsequent stable gallop in a robotic quadruped. The MIT Cheetah I, a planar quadruped platform for high-speed running, achieves these tasks with a speed of 3.2 m/s (Froude number of 2.1) on a treadmill. The controller benefits from clues from biological findings and it incorporates (1) a gait pattern modulation that imposes predefined gait patterns with a proprioceptive touchdown feedback, (2) tunable equilibrium-point foot-end trajectories for four limbs that intentionally modulate ground reaction forces, and (3) programmable leg compliance that provides instantaneous reflexes to leg–ground interaction. An inertial measurement unit sensor is integrated with the controller in order to regulate leg angles of attack at touchdown. We reduce the dimension of the control parameters which describe temporal/spatial characteristics of quadruped locomotion, and the values are tuned via dynamic simulation and then experiment. Given a pre-defined virtual leg compliance and a desired angle of attack of legs, the equilibrium-point foot-end trajectories and phase relationships between four legs for stable trot and gallop gaits are found independently. We propose a simple throw-and-catch gait transition strategy which connects two stable limit cycles, the trot and the gallop, by linearly varying control parameters during the transition period. Successful gait transition is achieved in both simulation and experiment. Comprehensive analysis on the characteristics of the MIT Cheetah I experimental trot-to-gallop transition is provided. The phase portraits imply that stable limit cycles are achieved with the proposed controller in both trot and gallop, which enables the trot-to-gallop gait transition at high speed.
ATRIAS is a human-scale 3D-capable bipedal robot designed to mechanically embody the spring-mass model for dynamic walking and running. To help bring the extensive work on this theoretical model further into practice, we present the design and validation of a spring-mass robot that can operate in real-world settings (i.e. off-tether and without planarizing restraints). We outline the mechanisms and design choices necessary to meet these specifications, particularly ATRIAS’ four-bar series-elastic leg design. We experimentally demonstrate the following robot capabilities, which are characteristics of the target model. 1) We present the robot’s physical capability for both grounded and aerial gaits, including planar walking and sustained hopping, while being more efficient than similarly gait-versatile bipeds. 2) The robot can be controlled by enforcing quantities derived from the simpler spring-mass model, such as leg angles and leg forces. 3) ATRIAS replicates the center-of-mass dynamics of human hopping and (novelly) walking, a key spring-mass model feature. Lastly, we present dynamically stable stepping in 3D without external support, demonstrating that this theoretical model has practical potential for real-world locomotion.
Rigid bodies, plastic impact, persistent contact, Coulomb friction, and massless limbs are ubiquitous simplifications introduced to reduce the complexity of mechanics models despite the obvious physical inaccuracies that each incurs individually. In concert, it is well known that the interaction of such idealized approximations can lead to conflicting and even paradoxical results. As robotics modeling moves from the consideration of isolated behaviors to the analysis of tasks requiring their composition, a mathematically tractable framework for building models that combine these simple approximations yet achieve reliable results is overdue. In this paper we present a formal hybrid dynamical system model that introduces suitably restricted compositions of these familiar abstractions with the guarantee of consistency analogous to global existence and uniqueness in classical dynamical systems. The hybrid system developed here provides a discontinuous but self-consistent approximation to the continuous (though possibly very stiff and fast) dynamics of a physical robot undergoing intermittent impacts. The modeling choices sacrifice some quantitative numerical efficiencies while maintaining qualitatively correct and analytically tractable results with consistency guarantees promoting their use in formal reasoning about mechanism, feedback control, and behavior design in robots that make and break contact with their environment.
This paper presents a Robust Constrained Learning-based Nonlinear Model Predictive Control (RC-LB-NMPC) algorithm for path-tracking in off-road terrain. For mobile robots, constraints may represent solid obstacles or localization limits. As a result, constraint satisfaction is required for safety. Constraint satisfaction is typically guaranteed through the use of accurate, a priori models or robust control. However, accurate models are generally not available for off-road operation. Furthermore, robust controllers are often conservative, since model uncertainty is not updated online. In this work our goal is to use learning to generate low-uncertainty, non-parametric models in situ. Based on these models, the predictive controller computes both linear and angular velocities in real-time, such that the robot drives at or near its capabilities while respecting path and localization constraints. Localization for the controller is provided by an on-board, vision-based mapping and navigation system enabling operation in large-scale, off-road environments. The paper presents experimental results, including over 5 km of travel by a 900 kg skid-steered robot at speeds of up to 2.0 m/s. The result is a robust, learning controller that provides safe, conservative control during initial trials when model uncertainty is high and converges to high-performance, optimal control during later trials when model uncertainty is reduced with experience.
We present a novel approach to perform fast probabilistic collision checking in high-dimensional configuration spaces to accelerate the performance of sampling-based motion planning. Our formulation stores the results of prior collision queries, and then uses such information to predict the collision probability for a new configuration sample. In particular, we perform an approximate k-NN (k-nearest neighbor) search to find prior query samples that are closest to the new query configuration. The new query sample’s collision status is then estimated according to the collision checking results of these prior query samples, based on the fact that nearby configurations are likely to have the same collision status. We use locality-sensitive hashing techniques with sub-linear time complexity for approximate k-NN queries. We evaluate the benefit of our probabilistic collision checking approach by integrating it with a wide variety of sampling-based motion planners, including PRM (Probabilistic roadmaps), lazyPRM, RRT Rapidly exploring random trees, and RRT*. Our method can improve these planners in various manners, such as accelerating the local path validation, or computing an efficient order for the graph search on the roadmap. Experiments on a set of benchmarks demonstrate the performance of our method, and we observe up to 2x speedup in the performance of planners on rigid and articulated robots.
Modeling physical human-robot interactions (pHRI) is important in studying human sensorimotor skills and designing human assistive and rehabilitation systems. One of the main challenges for modeling pHRI is the high dimensionality and complexity of human motion and its interactions with robots and the environment. We present an integrated physical-learning pHRI modeling framework with applications to the bikebot riding example. The modeling framework contains an integrated machine-learning-based model for high-dimensional limb motion with a physical-principle-based dynamic model for the human trunk and an interacted bicycle-like robot (bikebot). A new axial linear embedding algorithm is used to obtain the low-dimensional latent dynamics for highly redundant human limb movement. The integrated physical-learning model is then used to estimate human motion through an extended Kalman filter design without using any sensors attached to the limb segments. Extensive bikebot riding experiments are conducted to validate and demonstrate the integrated pHRI model. Comparison results with other machine-learning-based models are also presented to demonstrate the superior performance of the proposed modeling framework for bikebot riding.
The goal of this work is to develop computational models of social intelligence that enable robots to work side by side with humans, solving problems and achieving task goals through dialogue and collaborative manipulation. A defining problem of collaborative behavior in an embodied setting is the manner in which multiple agents make use of shared resources. In a situated dialogue, these resources include physical bottlenecks such as objects or spatial regions, and cognitive bottlenecks such as the speaking floor. For a robot to function as an effective collaborative partner with a human, it must be able to seize and yield such resources appropriately according to social expectations. We describe a general framework that uses timed Petri nets for the modeling and execution of robot speech, gaze, gesture, and manipulation for collaboration. The system dynamically monitors resource requirements and availability to control real-time turn-taking decisions over resources that are shared with humans, reasoning about different resource types independently. We evaluate our approach with an experiment in which our robot Simon performs a collaborative assembly task with 26 different human partners, showing that the multimodal reciprocal approach results in superior task performance, fluency, and balance of control.
Inspired by the aerial prowess of flying insects, we demonstrate that their robotic counterpart, an insect-scale flapping-wing robot, can mimic an aggressive maneuver seen in natural fliers— landing on a vertical wall. Such acrobatic movement differs from simple lateral maneuvers or hover, and therefore requires additional considerations in the control strategy. In this paper, we propose a single-loop adaptive tracking flight control suite designed with an emphasis on the ability to track dynamic trajectories, and an iterative learning control algorithm to account for unmodeled dynamics and systematic errors for improved landing accuracy. Magnets were chosen to enable attachment to the vertical surface due to their simplicity. The proposed controller was verified in a series of hovering and aggressive translational flights. Furthermore, we show that by learning from previous failed attempts, the biologically-inspired robot could successfully perch on a magnetic wall after eight learning iterations.
This paper presents a distributed, guidance and control algorithm for reconfiguring swarms composed of hundreds to thousands of agents with limited communication and computation capabilities. This algorithm solves both the optimal assignment and collision-free trajectory generation for robotic swarms, in an integrated manner, when given the desired shape of the swarm (without pre-assigned terminal positions). The optimal assignment problem is solved using a distributed auction assignment that can vary the number of target positions in the assignment, and the collision-free trajectories are generated using sequential convex programming. Finally, model predictive control is used to solve the assignment and trajectory generation in real time using a receding horizon. The model predictive control formulation uses current state measurements to resolve for the optimal assignment and trajectory. The implementation of the distributed auction algorithm and sequential convex programming using model predictive control produces the swarm assignment and trajectory optimization (SATO) algorithm that transfers a swarm of robots or vehicles to a desired shape in a distributed fashion. Once the desired shape is uploaded to the swarm, the algorithm determines where each robot goes and how it should get there in a fuel-efficient, collision-free manner. Results of flight experiments using multiple quadcopters show the effectiveness of the proposed SATO algorithm.
In this study, we propose an innovative driving method for a microrobot. By using acoustic levitation, the microrobot can be levitated from the glass substrate. We are able to achieve positioning accuracy of less than 1 μm, and the response speed and output force are also significantly improved. Silicon-based microrobots can be made into diverse shapes using deep reactive-ion etching (DRIE). Using custom-designed microrobots allows for the 3-D rotational control of a single bovine oocyte. Orientation with an accuracy of 1° and an average rotation velocity of 3 rad/s are achieved. This study contributes to the biotechnology. In the study of oocytes/embryos, manipulation is used for the enucleation, microinjection, and investigation of the characteristics of oocytes, such as the meiotic spindle and zona pellucida using PolScope. These studies and their clinical applications involve the three-dimensional (3-D) rotation of mammalian oocytes. The overall out-of-plane and in-plane rotations of the oocyte are demonstrated by using an acoustically levitated microrobot. In addition, by using this approach, it becomes much easier to manipulate the cell to investigate the characteristics of the single cell and analyze its mechanical properties.
Coverage is a fundamental problem in robotics, where one or more robots are required to visit each point in a target area at least once. Most previous work has concentrated on finding a coverage path that would minimize the coverage time. In this paper, we consider a new and more general version of the problem: adversarial coverage. Here, the robot operates in an environment that contains threats that might stop the robot. The objective is to cover the target area as quickly as possible, while minimizing the probability that the robot will be stopped before completing the coverage. This version of the problem has many real-world applications, from performing coverage missions in hazardous fields such as nuclear power plants, to surveillance of enemy forces in the battlefield and field demining. In this paper, we discuss the offline version of adversarial coverage, in which a map of the threats is given to the robot in advance. First, we formally define the adversarial coverage problem and present different optimization criteria used to evaluate coverage algorithms in adversarial environments. We show that finding an optimal solution to the adversarial coverage problem is
Vision-based place recognition is becoming an increasingly viable component of navigation systems for autonomous robots and personal aids. However, attaining robustness to variations in environmental conditions—such as time of day, weather and season—and camera viewpoint remains a major challenge. Featureless, sequence-based place recognition techniques have demonstrated promise, but often rely on long image sequences, manually-tuned parameters and exhaustive sequence match searching through multiple locations and image scales. In this paper, we address these deficiencies by implementing a condition-invariant, sequence-based place recognition algorithm suitable for networked environments, such as city streets, and routes with lateral platform shift, such as multiple-lane roads. We achieve this capability by augmenting the traditional 1D image database with a directed graph to describe the branching of contiguous sections of imagery at intersections. A particle filter is then used to efficiently explore these paths, as well as various lateral positions synthesized by rescaling imagery. Our proposed approach eliminates manual tuning of sequence length parameters, improves localization on branched routes, improves overall place recognition accuracy and coverage, and reduces computational requirements. We evaluated the new method against the original SeqSLAM and SMART algorithms on two day–night, road-based datasets and a summer–winter train dataset, where it attained superior precision-recall performance and coverage in all environments. Together, these contributions represent a significant step towards the provision of a robust, near parameter-free condition- and viewpoint-invariant visual place recognition capability for vehicles and robots.
This paper reports a comparative experimental evaluation of one non-model-based proportional derivative (PD) six-degree-of-freedom (6-DOF) controller and two model-based 6-DOF controllers designed to enable low-speed, neutrally buoyant, and fully actuated underwater vehicles to perform 6-DOF set-point regulation and trajectory tracking. We show analytically that the non-model-based PD controller provides locally asymptotically stable set-point regulation, and we show analytically that the model-based controllers provide locally asymptotically stable 6-DOF trajectory tracking. Numerical simulation studies are reported that corroborate the analytical stability results. We report the first comparative experimental evaluation of three different control algorithms for dynamic 6-DOF trajectory tracking of fully actuated underwater vehicles. Experimental results with the Johns Hopkins University remotely operated vehicle (JHU ROV) show that the model-based controllers’ mean absolute position and velocity tracking error is significantly smaller than the non-model-based PD controller for coupled maneuvers. The model-based controllers are shown to outperform the non-model-based controllers over a wide range of variations in the magnitude of derivative feedback gain. The velocity tracking error of the model-based controllers is shown to be on the same order of magnitude as the measurement error of the velocity sensing instrumentation.
We present a fundamental framework for organizing exploration, coverage, and surveillance by a swarm of robots with limited individual capabilities, based on triangulating an unknown environment with a multi-robot system. Locally, an individual triangle is easy for a single robot to manage and covers a small area; globally, the topology of the triangulation approximately captures the geometry of the entire environment. Combined, a multi-robot system can explore, map, navigate, and patrol. Algorithms can store information in triangles that the robots can read and write as they run their algorithms. This creates a physical data structure (PDS) that is both robust and versatile.
We study distributed approaches to triangulating an unknown, two-dimensional Euclidean space using a multi-robot network. The resulting PDS is a compact representation of the workspace, contains distributed knowledge of each triangle, encodes the dual graph of the triangulation, and supports reads and writes of auxiliary data. The ability to store and process this auxiliary information enables the simple robots to solve complex problems. We develop distributed algorithms for dual-graph navigation, patrolling, construction of a topological Voronoi tessellation, and location of the geodesic centers in non-convex regions. We provide theoretical performance guarantees for the quality of constructed triangulation and the connectivity of a dual graph in the triangulation. In addition, we show that the path lengths of the physical navigation are within a constant factor of the shortest-path Euclidean distance. We validate these theoretical results with simulations and experiments with a dozen or more robots.
This paper investigates the problem of confirming the identity of a candidate object (expected to be a target based on some crude visual clues) with a mobile robot equipped with visual sensing capabilities. We present a method whose main novelty is to mix localization of the robot relative to the candidate object and to confirm that it is the sought target. This twofold approach drastically reduces false positives. Identity confirmation with this twofold goal is modeled as a Partially-Observable Markov Decision Process, where the states are the cells of the space decomposition. It is solved using Stochastic Dynamic Programming with imperfect state information. A robotic system using this method has been implemented and tests have been carried out both in simulation and with a real robot. The experiments empirically validate the use of various metrics, and demonstrate their ability to perform well in different settings.
A new kind of underwater vehicle is developed by taking inspiration from cephalopods. Its actuation routine is scrutinized via a suitable model. Similar to octopuses and squids, these vehicles consist of an elastic, hollow shell capable of undergoing sequential stages of ingestion and ejection of ambient fluid, which is driven by the recursive inflation and deflation of the shell. The shell actively collapses, and in this way it expels water through a funnel; then it passively returns to the inflated shape, drawing ambient fluid into the cavity. By doing so, a pulsed-jet propulsion routine is performed that enables the vehicle to propel itself in water. Due to their soft nature, the actuation of these vehicles is largely dependent on the subtle management of the elastic response of the shell throughout the propulsion routine. A kinematic model of the actuation mechanism, thoroughly corroborated by experimental validation, is devised which elucidates the relationship between the active (collapse) and passive (refill) stages of the actuation. Upon association with the dynamics of the vehicle, this model permits the derivation of the generic performance profiles of this new kind of vehicle. It is acknowledged that, for given design specifications, an optimal swimming speed exists in coincidence with the coordinated operation between the crank mechanism driving the shell contraction and the onset of elastic energy, which determines the speed of inflation of the shell. These results are invaluable in the definition of rigorous design criteria and derivation of ad-hoc control laws for a new breed of optimized soft-bodied, pulsed-jet, unmanned underwater vehicles.
Collision checking is considered to be the most expensive computational bottleneck in sampling-based motion planning algorithms. We introduce a simple procedure that theoretically eliminates this bottleneck and significantly reduces collision-checking time in practice in several test scenarios. Whenever a point is collision checked in the normal (expensive) way, we store a lower bound on that point’s distance to the nearest obstacle. The latter is called a "safety certificate" and defines a region of the search space that is guaranteed to be collision-free. New points may forgo collision checking whenever they are located within a safety certificate of an old point. Testing the latter condition is accomplished during the nearest-neighbor search that is already part of most sampling-based motion planning algorithms. As more and more points are sampled, safety certificates asymptotically cover the search space and the amortized complexity of (normal, expensive) collision checking becomes negligible with respect to the overall runtime of sampling-based motion planning algorithms. Indeed, the expected fraction of points requiring a normal collision check approaches zero, in the limit, as the total number of points approaches infinity. A number of extensions to the basic idea are presented. Experiments with a number of proof-of-concept implementations demonstrate that using safety certificates can improve the performance of sampling-based motion planning algorithms in practice.
This paper presents visual-inertial datasets collected on-board a micro aerial vehicle. The datasets contain synchronized stereo images, IMU measurements and accurate ground truth. The first batch of datasets facilitates the design and evaluation of visual-inertial localization algorithms on real flight data. It was collected in an industrial environment and contains millimeter accurate position ground truth from a laser tracking system. The second batch of datasets is aimed at precise 3D environment reconstruction and was recorded in a room equipped with a motion capture system. The datasets contain 6D pose ground truth and a detailed 3D scan of the environment. Eleven datasets are provided in total, ranging from slow flights under good visual conditions to dynamic flights with motion blur and poor illumination, enabling researchers to thoroughly test and evaluate their algorithms. All datasets contain raw sensor measurements, spatio-temporally aligned sensor data and ground truth, extrinsic and intrinsic calibrations and datasets for custom calibrations.
Planetary excavator robots face unique and extreme engineering constraints relative to terrestrial counterparts. In space missions mass is always at a premium because it is the main driver behind launch costs. Lightweight operation, due to low mass and reduced gravity, hinders excavation and mobility by reducing the forces a robot can effect on its environment. This work shows that there is a quantifiable, non-dimensional threshold that distinguishes the regimes of lightweight and nominal excavation. This threshold is crossed at lower weights for continuous excavators (e.g. bucket-wheels) than discrete scrapers. This research introduces novel experimentation that for the first time subjects excavators to gravity offload (a cable pulls up on the robot with five-sixths its weight, to simulate lunar gravity) while they dig. A 300 kg excavator robot offloaded to 1/6 g successfully collects 0.5 kg/s using a bucket-wheel, with no discernible effect on mobility. For a discrete scraper of the same weight, production rapidly declines as rising excavation resistance stalls the robot. These experiments suggest caution in interpreting low-gravity performance predictions based solely on testing in Earth gravity. Experiments were conducted in GRC-1, a washed industrial silica-sand devoid of agglutinates and of the sub-75-micron basaltic fines that make up 40% of lunar regolith. The important dangers related to dust are thus not directly addressed. The achieved densities for experimentation are 1640 kg/m3 (very loose/loose) and 1720 kg/m3 (medium dense). This work develops a novel robotic bucket-wheel excavator, featuring unique direct transfer from bucket-wheel to dump bed as a solution to material transfer difficulties identified in past literature.
Solving problems combining task and motion planning requires searching across a symbolic search space and a geometric search space. Because of the semantic gap between symbolic and geometric representations, symbolic sequences of actions are not guaranteed to be geometrically feasible. This compels us to search in the combined search space, in which frequent backtracks between symbolic and geometric levels make the search inefficient. We address this problem by guiding symbolic search with rich information extracted from the geometric level through culprit detection mechanisms.
Mobile robots are increasingly populating our human environments. To interact with humans in a socially compliant way, these robots need to understand and comply with mutually accepted rules. In this paper, we present a novel approach to model the cooperative navigation behavior of humans. We model their behavior in terms of a mixture distribution that captures both the discrete navigation decisions, such as going left or going right, as well as the natural variance of human trajectories. Our approach learns the model parameters of this distribution that match, in expectation, the observed behavior in terms of user-defined features. To compute the feature expectations over the resulting high-dimensional continuous distributions, we use Hamiltonian Markov chain Monte Carlo sampling. Furthermore, we rely on a Voronoi graph of the environment to efficiently explore the space of trajectories from the robot’s current position to its target position. Using the proposed model, our method is able to imitate the behavior of pedestrians or, alternatively, to replicate a specific behavior that was taught by tele-operation in the target environment of the robot. We implemented our approach on a real mobile robot and demonstrated that it is able to successfully navigate in an office environment in the presence of humans. An extensive set of experiments suggests that our technique outperforms state-of-the-art methods to model the behavior of pedestrians, which also makes it applicable to fields such as behavioral science or computer graphics.
This work addresses the development and application of a novel approach, called sparser relative bundle adjustment (SRBA), which exploits the inherent flexibility of the relative bundle adjustment (RBA) framework to devise a continuum of strategies, ranging from RBA with linear graphs to classic bundle adjustment (BA) in global coordinates, where submapping with local maps emerges as a natural intermediate solution. This method leads to graphs that can be optimized in bounded time even at loop closures, regardless of the loop length. Furthermore, it is shown that the pattern in which relative coordinate variables are defined among keyframes has a significant impact on the graph optimization problem. By using the proposed scheme, optimization can be done more efficiently than in standard RBA, allowing the optimization of larger local maps for any given maximum computational cost. The main algorithms involved in the graph management, along with their complexity analyses, are presented to prove their bounded-time nature. One key advance of the present work is the demonstration that, under mild assumptions, the spanning trees for every single keyframe in the map can be incrementally built by a constant-time algorithm, even for arbitrary graph topologies. We validate our proposal within the scope of visual stereo simultaneous localization and mapping (SLAM) by developing a complete system that includes a front-end that seamlessly integrates several state-of-the-art computer vision techniques such as ORB features and bag-of-words, along with a decision scheme for keyframe insertion and a SRBA-based back-end that operates as graph optimizer. Finally, a set of experiments in both indoor and outdoor conditions is presented to test the capabilities of this approach. Open-source implementations of the SRBA back-end and the stereo front-end have been released online.
This paper documents a large scale, long-term autonomy dataset for robotics research collected on the University of Michigan’s North Campus. The dataset consists of omnidirectional imagery, 3D lidar, planar lidar, GPS, and proprioceptive sensors for odometry collected using a Segway robot. The dataset was collected to facilitate research focusing on long-term autonomous operation in changing environments. The dataset is composed of 27 sessions spaced approximately biweekly over the course of 15 months. The sessions repeatedly explore the campus, both indoors and outdoors, on varying trajectories, and at different times of the day across all four seasons. This allows the dataset to capture many challenging elements including: moving obstacles (e.g. pedestrians, bicyclists and cars), changing lighting, varying viewpoint, seasonal and weather changes (e.g. falling leaves and snow), and long-term structural changes caused by construction projects. To further facilitate research, we also provide ground-truth pose for all sessions in a single frame of reference.
This paper explores the benefits of using multiple gaits in a single robot. Inspired by nature, where humans and animals use different gaits to increase their energetic economy, we analyzed how increasing speed affects the choice of gait, and how the choice of gait influences optimal speed. To this end, we used optimal control as a tool to identify motions that minimize the cost of transport of two detailed models: a planar biped and a planar quadruped. Both of these models are actuated with high compliance series elastic actuators that enable a rich set of natural dynamics. These models have damping in their springs, feet with mass, and realistic limitations on actuator torques and velocities. They therefore serve as an intermediary between past simpler models and hardware. We discovered optimal motions with an established multiple shooting implementation that relies on pre-defined contact sequences, and with a direct collocation implementation in which the footfall pattern was an outcome of the optimization. Both algorithms confirmed findings from biology. For both models, changing gaits as speed varies leads to greatly increased energetic economy. For bipeds, the optimal gaits were walking at low speeds, grounded running at intermediate speeds, and running at high speeds. For quadrupeds, the optimal gaits were four-beat walking at low speeds and trotting at intermediate speeds. At high speeds, galloping and trotting were the best gaits, with nearly equal performance. We found that the transition between gaits was primarily driven by damping losses and negative actuator work, with collisions playing a relatively small role.
This paper develops a theoretical framework for grasping objects using customized effectors that have curved contact surfaces, and presents its application to the problem of robotic whole-arm grasping. We present a collection of immobilizing grasps and cages that can effectively restrain the mobility of a wide range of object shapes including polyhedra. Each of the grasps or cages is formed by at most three effectors with appropriate contact surfaces in contrast to customary point fingertips. We also discuss the morphology of the curved contact surfaces that can realize the grasps and cages; the surfaces can simply be planar, cylindrical, or spherical. Stable grasps are obtained by simple motion planning and control. Our theory is based on a conservative assumption that all contacts are frictionless, rigid, unilateral. Finally, we present a robotic system, comprised of a software suite and a modular reconfigurable manipulator outfitted with exchangeable end-effectors and arm links, demonstrating the theory and our approach to whole-arm grasping.
This paper introduces a method to minimize norms on nonlinear trajectory sensitivities during open-loop trajectory optimization. Specifically, we derive new parametric sensitivity terms that measure the variation in nonlinear (continuous-time) trajectories due to variations in model parameters, and hybrid sensitivities, which account for variations in trajectory caused by sudden transitions from nominal dynamics to alternative dynamic modes. We adapt continuous trajectory optimization to minimize these sensitivities while only minimally changing a nominal trajectory. We provide appended states, cost, and linearizations, required so that existing open-loop optimization methods can generate minimally sensitive feedforward trajectories. Although there are several applications for sensitivity optimization, this paper focuses on robot motion planning, where popular sample-based planners rely on local trajectory generation to expand tree/graph structures. While such planners often use stochastic uncertainty propagation to model and reduce uncertainty, this paper shows that trajectory uncertainty can be reduced by minimizing first-order sensitivities. Simulated vehicle examples show parametric sensitivity optimization generates trajectories optimally insensitive to parametric model uncertainty. Similarly, minimizing hybrid sensitivities reduces uncertainty in crossing mobility hazards (e.g. rough terrain, sand, ice). Examples demonstrate the process yields a planner that uses approximate hazard models to automatically and optimally choose when to avoid hazardous terrain and when controls can be adjusted to traverse hazards with reduced uncertainty. Sensitivity optimization offers a simple alternative to stochastic simulation and complicated uncertainty modeling for nonlinear systems.
This paper presents a relaxed definition of hover for multicopters with propellers pointing in a common direction. These solutions are found by requiring that the multicopter remain substantially in one position, and that the solutions be constant when expressed in a coordinate system attached to the vehicle. The vehicle’s angular velocity is then shown to be either zero or parallel to gravity. The controllability of a vehicle’s attitude about these solutions is then investigated. These relaxed hover solutions may be applied as an algorithmic failsafe, allowing, for example, a quadrocopter to fly despite the complete loss of one, two, or three of its propellers. Experimental results validate the quadrocopter failsafe for two types of failure (a single propeller and two opposing propellers failing), and a nonlinear simulation validates the remaining two types of failure (two adjacent and three propellers failing). The relaxed hover solutions are also shown to allow a multicopter to maintain flight in spite of extreme center of mass offsets. Finally, the design and experimental validation of three novel vehicles is presented.
The stable, quantization-error noise-free, rendering of high-stiffness dynamics can be challenging using impedance-type haptic displays. In this paper we examine a canonical, one degree of freedom, haptic display rendering a virtual spring and damper, including the effects of the device and human dynamics, sampling, position quantization, time delay, and the low-pass filter operating on the device velocity estimate. We construct various stability and quantization-error regions as a function of the system parameters and show the necessary trade-offs that occur between them. Although we apply the quantization-error analysis to virtual spring and damper rendering, it applies to a general virtual environment. We present sufficiency for quantization-error passivity, necessity for no malicious-touch limit cycles, and necessity for no uncoupled-touch limit cycles. Using these results, aided by the presented supplementary code, we find control parameters to render the largest, renderable, virtual stiffness for a given haptic display. The analytical results are experimentally verified using a Phantom Premium 1.5 haptic device.
This paper presents a robotic manipulation system capable of autonomously positioning a multi-segment soft fluidic elastomer robot in three dimensions. Specifically, we present an extremely soft robotic manipulator morphology that is composed entirely from low durometer elastomer, powered by pressurized air, and designed to be both modular and durable. To understand the deformation of a single arm segment, we develop and experimentally validate a static deformation model. Then, to kinematically model the multi-segment manipulator, we use a piece-wise constant curvature assumption consistent with more traditional continuum manipulators. In addition, we define a complete fabrication process for this new manipulator and use this process to make multiple functional prototypes. In order to power the robot’s spatial actuation, a high capacity fluidic drive cylinder array is implemented, providing continuously variable, closed-circuit gas delivery. Next, using real-time data from a vision system, we develop a processing and control algorithm that generates realizable kinematic curvature trajectories and controls the manipulator’s configuration along these trajectories. Lastly, we experimentally demonstrate new capabilities offered by this soft fluidic elastomer manipulation system such as entering and advancing through confined three-dimensional environments as well as conforming to goal shape-configurations within a sagittal plane under closed-loop control.
Dynamic environments have obstacles that unpredictably appear, disappear, or move. We present the first sampling-based replanning algorithm that is asymptotically optimal and single-query (designed for situation in which a priori offline computation is unavailable). Our algorithm, RRTX, refines and repairs the same search-graph over the entire duration of navigation (in contrast to previous single-query replanning algorithms that prune and then regrow some or all of the search-tree). Whenever obstacles change and/or the robot moves, a graph rewiring cascade quickly remodels the existing search-graph and repairs its shortest-path-to-goal sub-tree to reflect the new information. Both graph and tree are built directly in the robot’s state-space; thus, the resulting plan(s) respect the kinematics of the robot and continue to improve during navigation. RRTX is probabilistically complete and makes no distinction between local and global planning, yet it reacts quickly enough for real-time high-speed navigation through unpredictably changing environments. Low information transfer time is essential for enabling RRTX to react quickly in dynamic environments; we prove that the information transfer time required to inform a graph of size n about an -cost decrease is O(n log n) for RRTX—faster than other current asymptotically optimal single-query algorithms (we prove RRT* is
This paper presents a method for one-shot learning of dexterous grasps and grasp generation for novel objects. A model of each grasp type is learned from a single kinesthetic demonstration and several types are taught. These models are used to select and generate grasps for unfamiliar objects. Both the learning and generation stages use an incomplete point cloud from a depth camera, so no prior model of an object shape is used. The learned model is a product of experts, in which experts are of two types. The first type is a contact model and is a density over the pose of a single hand link relative to the local object surface. The second type is the hand-configuration model and is a density over the whole-hand configuration. Grasp generation for an unfamiliar object optimizes the product of these two model types, generating thousands of grasp candidates in under 30 seconds. The method is robust to incomplete data at both training and testing stages. When several grasp types are considered the method selects the highest-likelihood grasp across all the types. In an experiment, the training set consisted of five different grasps and the test set of 45 previously unseen objects. The success rate of the first-choice grasp is 84.4% or 77.7% if seven views or a single view of the test object are taken, respectively.
We present new parallel algorithms that solve continuous-state partially observable Markov decision process (POMDP) problems using the GPU (gPOMDP) and a hybrid of the GPU and CPU (hPOMDP). We choose the Monte Carlo value iteration (MCVI) method as our base algorithm and parallelize this algorithm using the multi-level parallel formulation of MCVI. For each parallel level, we propose efficient algorithms to utilize the massive data parallelism available on modern GPUs. Our GPU-based method uses the two workload distribution techniques, compute/data interleaving and workload balancing, in order to obtain the maximum parallel performance at the highest level. Here we also present a CPU–GPU hybrid method that takes advantage of both CPU and GPU parallelism in order to solve highly complex POMDP planning problems. The CPU is responsible for data preparation, while the GPU performs Monte Cacrlo simulations; these operations are performed concurrently using the compute/data overlap technique between the CPU and GPU. To the best of the authors’ knowledge, our algorithms are the first parallel algorithms that efficiently execute POMDP in a massively parallel fashion utilizing the GPU or a hybrid of the GPU and CPU. Our algorithms outperform the existing CPU-based algorithm by a factor of 75–99 based on the chosen benchmark.
This paper presents a systematic approach for the design of continuous-time controllers to robustly and exponentially stabilize periodic orbits of hybrid dynamical systems arising from bipedal walking. A parameterized family of continuous-time controllers is assumed so that (1) a periodic orbit is induced for the hybrid system, and (2) the orbit is invariant under the choice of controller parameters. Properties of the Poincaré map and its first- and second-order derivatives are used to translate the problem of exponential stabilization of the periodic orbit into a set of bilinear matrix inequalities (BMIs). A BMI optimization problem is then set up to tune the parameters of the continuous-time controller so that the Jacobian of the Poincaré map has its eigenvalues in the unit circle. It is also shown how robustness against uncertainty in the switching condition of the hybrid system can be incorporated into the design problem. The power of this approach is illustrated by finding robust and stabilizing continuous-time feedback laws for walking gaits of two underactuated 3D bipedal robots.
The goal of this work is to develop a soft-robotic manipulation system that is capable of autonomous, dynamic, and safe interactions with humans and its environment. First, we develop a dynamic model for a multi-body fluidic elastomer manipulator that is composed entirely from soft rubber and subject to the self-loading effects of gravity. Then, we present a strategy for independently identifying all of the unknown components of the system; these are the soft manipulator, its distributed fluidic elastomer actuators, as well as the drive cylinders that supply fluid energy. Next, using this model and trajectory-optimization techniques we find locally-optimal open-loop policies that allow the system to perform dynamic maneuvers we call grabs. In 37 experimental trials with a physical prototype, we successfully perform a grab 92% of the time. Last, we introduce the idea of static bracing for a soft elastomer arm and discuss how forming environmental braces might be an effective manipulation strategy for this class of robots. By studying such an extreme example of a soft robot, we can begin to solve hard problems inhibiting the mainstream use of soft machines.
In robotics, the use of a classification framework which produces scores with inappropriate confidences will ultimately lead to the robot making dangerous decisions. In order to select a framework which will make the best decisions, we should pay careful attention to the ways in which it generates scores. Precision and recall have been widely adopted as canonical metrics to quantify the performance of learning algorithms, but for robotics applications involving mission-critical decision making, good performance in relation to these metrics is insufficient. We introduce and motivate the importance of a classifier’s introspective capacity: the ability to associate an appropriate assessment of confidence with any test case. We propose that a key ingredient for introspection is a framework’s potential to increase its uncertainty with the distance between a test datum its training data. We compare the introspective capacities of a number of commonly used classification frameworks in both classification and detection tasks, and show that better introspection leads to improved decision making in the context of tasks such as autonomous driving or semantic map generation.
Most robotic grasping research focuses on objects that are either not large in comparison to the gripper or have small graspable features; however, there are important applications that involve large flat or gently curved surfaces. Examples include robots that grasp the solar panels of space craft, handle large panels in manufacturing, or climb or perch on surfaces. We present a solution for grasping such surfaces consisting of groups of tiles coated with a controllable gecko-inspired adhesive. The tiles are loaded with two sets of tendons: one for distributing the forces evenly while grasping and the other for release. The gripper is passive and can attach and detach with little effort so that it does not disturb either the robot or the object to be grasped. The maximum gripping force in the normal direction can be over 1000 times greater than the required detaching force. The gripper is also fast, allowing a flying quadrotor to attach to a surface milliseconds after the tiles make contact. We present a model of the gripping mechanism and use the model to design the layout of the tiles to best support anticipated normal and tangential loads.
This paper proposes a task-oriented grasp quality metric based on distribution of task disturbance, which could be used to search for a grasp that covers the most significant part of the disturbance distribution. Rather than using a uniformly distributed task wrench space, this paper models a manipulation task with a non-parametric statistical distribution model built from the disturbance data captured during the task demonstrations. The grasp resulting from maximizing the proposed grasp quality criterion is prone to increasing the coverage of most frequent disturbances. To reduce the computational complexity of the search in a high-dimensional robotic hand configuration space, as well as to avoid the correspondence problem, the candidate grasps are computed from a reduced configuration space that is confined by a set of given thumb placements and thumb directions. The proposed approach has been tested both in simulation and on a real robotic system. In simulation, the approach was validated with a Barrett hand and a Shadow hand in several manipulation tasks. Experiments on a physical robotic platform verified the consistency between the proposed grasp metric and the success rate.
This paper proposes a self-assembling soft modular matter, called SoftCubes, where soft-bodied modules are disassembled into a flexible string by an external tension and self-assemble into the preprogrammed three-dimensional (3D) shape. The developed soft modular matter has three main design features. Firstly, entire modules of the 3D shape are serially connected. Such a structure allows all the modules to be disassembled by external tension. Secondly, the outer body of the modules and the connecting parts are made of soft stretchable elastomer. Due to the soft body of the modules, after disassembling, the serially connected modules become a highly flexible and soft string, and have an extreme shape adaptation capability. Also, if the external tension is removed, the preprogrammed 3D shape is recovered by the elastic restoring force of soft-bodied modules. Finally, embedded small permanent magnets induce magnetic self-assembling forces and maintain a mechanical robustness of the 3D shape of module assembly. Due to the magnetic self-assembly, the soft modules are precisely aligned with neighboring modules in a lattice structure. The paper also presents an algorithm to generate the serial connection path of modules for creating a given 3D shape. Various physical interactions between self-assembling module prototypes are visualized in two-dimensional motion tracking experiments. Finally, the shape reconfiguration ability of soft modular matter is demonstrated. SoftCubes is a new class of programmable modular matter where shape memory ability is embedded in the structure, and shows a physical implementation of various 3D shapes with a high resolution and a high scalability.
In this paper, we introduce an indirect pushing based technique for automated micromanipulation of biological cells. In indirect pushing, an optically trapped glass bead pushes a freely diffusing intermediate bead that in turn pushes a freely diffusing target cell towards a desired goal. Some cells can undergo significant changes in their behaviors as a result of direct exposure to a laser beam. Indirect pushing eliminates this problem by minimizing the exposure of the cell to the laser beam. We report an automated feedback planning algorithm that combines three motion maneuvers, namely, push, align, and backup for micromanipulation of cells. We have developed a dynamics based simulation model of indirect pushing dynamics and also identified parameters of measurement noise using physical experiments. We present an optimization-based approach for automated tuning of planner parameters to enhance its robustness. Finally, we have tested the developed planner using our optical tweezers physical setup and carried out a detailed analysis of the experimental results. The developed approach can be utilized in biological experiments for studying collective cell migration by accurately arranging the cells in arrays without exposing them to a laser beam.
In this paper, we consider automatic computation of optimal control strategies for a robot interacting with a set of independent uncontrollable agents in a graph-like environment. The mission specification is given as a syntactically co-safe Linear Temporal Logic formula over some properties that hold at the vertices of the environment. The robot is assumed to be a deterministic transition system, while the agents are probabilistic Markov models. The goal is to control the robot such that the probability of satisfying the mission specification is maximized. We propose a computationally efficient incremental algorithm based on the fact that temporal logic verification is computationally cheaper than synthesis. We present several case studies where we compare our approach to the classical non-incremental approach in terms of computation time and memory usage.
Research over the past several decades has elucidated some of the mechanisms behind high speed, highly efficient, and robust locomotion in insects such as cockroaches. Roboticists have used this information to create biologically inspired machines capable of running, jumping, and climbing robustly over a variety of terrains. To date, little work has been done to develop an at-scale insect-inspired robot capable of similar feats due to challenges in fabrication, actuation, and electronics integration for a centimeter-scale device. This paper addresses these challenges through the design, fabrication, and control of a 1.27 g walking robot, the Harvard Ambulatory MicroRobot (HAMR). The current design is manufactured using a method inspired by pop-up books that enables fast and repeatable assembly of the miniature walking robot. Methods to drive HAMR at low and high speeds are presented, resulting in speeds up to 0.44 m/s (10.1 body lengths per second) and the ability to maneuver and control the robot along desired trajectories.
When fusing visual and inertial measurements for motion estimation, each measurement’s sampling time must be precisely known. This requires knowledge of the time offset that inevitably exists between the two sensors’ data streams. The first contribution of this work is an online approach for estimating this time offset, by treating it as an additional state variable to be estimated along with all other variables of interest (inertial measurement unit (IMU) pose and velocity, biases, camera-to-IMU transformation, feature positions). We show that this approach can be employed in pose-tracking with mapped features, in simultaneous localization and mapping, and in visual–inertial odometry. The second main contribution of this paper is an analysis of the identifiability of the time offset between the visual and inertial sensors. We show that the offset is locally identifiable, except in a small number of degenerate motion cases, which we characterize in detail. These degenerate cases are either (i) cases known to cause loss of observability even when no time offset exists, or (ii) cases that are unlikely to occur in practice. Our simulation and experimental results validate these theoretical findings, and demonstrate that the proposed approach yields high-precision, consistent estimates, in scenarios involving either known or unknown features, with both constant and time-varying offsets.
In this paper, a distributed controller–observer schema for tracking control of the centroid and of the relative formation of a multi-robot system with first-order dynamics is presented. Each robot of the team uses a distributed observer to estimate the overall system state and a motion control strategy for tracking control of time-varying centroid and formation. Proof of the overall convergence of the controller–observer schema for different kinds of connection topologies, as well as for the cases of unsaturated and saturated control inputs is presented. In particular, the solution is proven to work in the case of strongly connected non-switching topologies and in the case of balanced strongly connected switching topologies. In order to complete the work, the approach is validated by experimental tests with a team of five wheeled mobile robots.
This paper presents the application of operational space control based on hierarchical task optimization for quadrupedal locomotion. We show how the behavior of a complex robotic machine can be described by a simple set of least squares problems with different priorities for motion, torque, and force optimization. Using projected dynamics of floating base systems with multiple contact points, the optimization dimensionality can be reduced or decoupled such that the formulation is purely based on the inversion of kinematic system properties. The present controller is extensively tested in various experiments using the fully torque controllable quadrupedal robot StarlETH. The load distribution is optimized for static walking gaits to improve contact stability and/or actuator efficiency under various terrain conditions. This is augmented with simultaneous joint position and torque limitations as well as with an interpolation method to ensure smooth contact transitions. The same control structure is further used to stabilize dynamic trotting gaits under significant external disturbances such as uneven ground or pushes. To the best of our knowledge, this work is the first documentation of static and dynamic locomotion with pure task-space inverse dynamics (no joint position feedback) control.
Hierarchical least-square optimization is often used in robotics to inverse a direct function when multiple incompatible objectives are involved. Typical examples are inverse kinematics or dynamics. The objectives can be given as equalities to be satisfied (e.g. point-to-point task) or as areas of satisfaction (e.g. the joint range). This paper proposes a complete solution to solve multiple least-square quadratic problems of both equality and inequality constraints ordered into a strict hierarchy. Our method is able to solve a hierarchy of only equalities 10 times faster than the iterative-projection hierarchical solvers and can consider inequalities at any level while running at the typical control frequency on whole-body size problems. This generic solver is used to resolve the redundancy of humanoid robots while generating complex movements in constrained environments.
This paper extends the use of virtual constraints and hybrid zero dynamics (HZD), a successful control strategy for point-foot bipeds, to the design of controllers for planar curved foot bipeds. Although the rolling contact constraint at the foot–ground interface increases complexity somewhat, the measure of local stability remains a function of configuration only, and a closed-form solution still determines the existence of a periodic orbit. The formulation is validated in experiment using the planar five-link biped ERNIE. While gaits designed for point feet yielded stable walking when ERNIE was equipped with curved feet, errors in both desired speed and joint tracking were significantly larger than for gaits designed for the correct radius curved feet. Thus, HZD-based control of this biped is shown to be robust to some modeling error in the foot radius, but at the same time, to require consideration of foot radius to achieve predictably reliable walking gaits. Additionally, under HZD-based control, this biped walked with lower specific energetic cost of transport and joint tracking errors for matched curved foot gait design and hardware compared to matched point-foot gait design and hardware.
We address the problem of controlling a noisy differential drive mobile robot such that the probability of satisfying a specification given as a bounded linear temporal logic formula over a set of properties at the regions in the environment is maximized. We assume that the vehicle can determine its precise initial position in a known map of the environment. However, motivated by practical limitations, we assume that the vehicle is equipped with noisy actuators and, during its motion in the environment, it can only measure the angular velocity of its wheels using limited accuracy incremental encoders. Assuming the duration of the motion is finite, we map the measurements to a Markov decision process (MDP). We use recent results in statistical model checking to obtain an MDP control policy that maximizes the probability of satisfaction. We translate this policy to a vehicle feedback control strategy and show that the probability that the vehicle satisfies the specification in the environment is bounded from below by the probability of satisfying the specification on the MDP. We illustrate our method with simulations and experimental results.
This work investigates the pose graph optimization problem, which arises in maximum likelihood approaches to simultaneous localization and mapping (SLAM). State-of-the-art approaches have been demonstrated to be very efficient in medium- and large-sized scenarios; however, their convergence to the maximum likelihood estimate heavily relies on the quality of the initial guess. We show that, in planar scenarios, pose graph optimization has a very peculiar structure. The problem of estimating robot orientations from relative orientation measurements is a quadratic optimization problem (after computing suitable regularization terms); moreover, given robot orientations, the overall optimization problem becomes quadratic. We exploit these observations to design an approximation of the maximum likelihood estimate, which does not require the availability of an initial guess. The approximation, named LAGO (Linear Approximation for pose Graph Optimization), can be used as a stand-alone tool or can bootstrap state-of-the-art techniques, reducing the risk of being trapped in local minima. We provide analytical results on existence and sub-optimality of LAGO, and we discuss the factors influencing its quality. Experimental results demonstrate that LAGO is accurate in common SLAM problems. Moreover, it is remarkably faster than state-of-the-art techniques, and is able to solve very large-scale problems in a few seconds.
This paper presents a solution to the problem of self-organized aggregation of embodied robots that requires no arithmetic computation. The robots have no memory and are equipped with one binary sensor, which informs them whether or not there is another robot in their line of sight. It is proven that the sensor needs to have a sufficiently long range; otherwise aggregation cannot be guaranteed, irrespective of the controller used. The optimal controller is found by performing a grid search over the space of all possible controllers. With this controller, robots rotate on the spot when they perceive another robot, and move backwards along a circular trajectory otherwise. This controller is proven to always aggregate two simultaneously moving robots in finite time, an upper bound for which is provided. Simulations show that the controller also aggregates at least 1000 robots into a single cluster consistently. Moreover, in 30 experiments with 40 physical e-puck robots, 98.6% of the robots aggregated into one cluster. The results obtained have profound implications for the implementation of multi-robot systems at scales where conventional approaches to sensing and information processing are no longer applicable.
Many applications of autonomy are significantly complicated by the need for wireless networking, with challenges including scalability and robustness. Radio accomplishes this in a complex environment, but suffers from rapid signal strength variation and attenuation typically much worse than free space loss. In this paper, we propose and test algorithms to autonomously discover the connectivity area for a base station in an unknown environment using an average of received signal strength (RSS) values and a RSS threshold to delineate the goodness of the channel. We combine region decomposition and RSS sampling to cast the problem as an efficient graph search. The nominal RSS in a sampling region is obtained by averaging local RSS samples to reduce the small-scale fading variation. The RSS gradient is exploited during exploration to develop an efficient approach for discovery of the base station connectivity boundary in an unknown environment. Indoor and outdoor experiments demonstrate the proposed techniques. The results can be used for sensing and collaborative autonomy, building base station coverage maps in unknown environments, and facilitating multi-hop relaying to a base station.
This paper addresses a navigation problem for a certain type of simple mobile robot modeled as a point moving in the plane. The only requirement on the robot is that it must be able to translate in a desired direction, with bounded angular error (measured in a global reference frame), until it reaches the nearest obstacle in its motion direction. One straightforward realization of this capability might use a noisy compass and a contact sensor. We present a planning algorithm that enables such a robot to navigate reliably through its environment. The algorithm constructs a directed graph in which each node is labeled with a subset of the environment boundary. Each edge of the graph is labeled with a sequence of actions that can move the robot from any location in one such set to some location in the other set. We use a variety of local planners to generate the edges, including a "corner-finding" technique that allows the robot to travel to and localize itself at a convex vertex of the environment boundary. The algorithm forms complete plans by searching the resulting graph. We have implemented this algorithm and present results from both simulation and a proof-of-concept physical realization.
This paper presents a general method for planning collision-free whole-body walking motions for humanoid robots. First, we present a randomized algorithm for constrained motion planning, that is used to generate collision-free statically balanced paths solving manipulation tasks. Then, we show that dynamic walking makes humanoid robots small-space controllable. Such a property allows to easily transform collision-free statically balanced paths into collision-free dynamically balanced trajectories. It leads to a sound algorithm which has been applied and evaluated on several problems where whole-body planning and walk are needed, and the results have been validated on a real HRP-2 robot.
Action prediction and fluidity are key elements of human–robot teamwork. If a robot’s actions are hard to understand, it can impede fluid human–robot interaction. Our goal is to improve the clarity of robot motion by making it more human-like. We present an algorithm that autonomously synthesizes human-like variants of an input motion. Our approach is a three-stage pipeline. First we optimize motion with respect to spatiotemporal correspondence (STC), which emulates the coordinated effects of human joints that are connected by muscles. We present three experiments that validate that our STC optimization approach increases human-likeness and recognition accuracy for human social partners. Next in the pipeline, we avoid repetitive motion by adding variance, through exploiting redundant and underutilized spaces of the input motion, which creates multiple motions from a single input. In two experiments we validate that our variance approach maintains the human-likeness from the previous step, and that a social partner can still accurately recognize the motion’s intent. As a final step, we maintain the robot’s ability to interact with its world by providing it the ability to satisfy constraints. We provide experimental analysis of the effects of constraints on the synthesized human-like robot motion variants.
We describe an integrated strategy for planning, perception, state estimation and action in complex mobile manipulation domains based on planning in the belief space of probability distributions over states using hierarchical goal regression (pre-image back-chaining). We develop a vocabulary of logical expressions that describe sets of belief states, which are goals and subgoals in the planning process. We show that a relatively small set of symbolic operators can give rise to task-oriented perception in support of the manipulation goals. An implementation of this method is demonstrated in simulation and on a real PR2 robot, showing robust, flexible solution of mobile manipulation problems with multiple objects and substantial uncertainty.
This paper presents an integrated motion planning and control framework that enables balancing mobile robots to grace-fully navigate human environments. A palette of controllers called motion policies is designed such that balancing mobile robots can achieve fast, graceful motions in small, collision-free domains of the position space. The domains determine the validity of a motion policy at any point in the robot’s position state space. An automatic instantiation procedure that generates a motion policy library by deploying motion policies from a palette on a map of the environment is presented. A gracefully prepares relationship that guarantees valid compositions of motion policies to produce overall graceful motion is introduced. A directed graph called the gracefully prepares graph is used to represent all valid compositions of motion policies in the motion policy library. The navigation tasks are achieved by planning in the space of these gracefully composable motion policies. In this work, Dijsktra’s algorithm is used to generate a single-goal optimal motion policy tree, and its variant is used to rapidly replan the optimal motion policy tree in the presence of dynamic obstacles. A hybrid controller is used as a supervisory controller to ensure successful execution of motion policies and also successful switching between them. The integrated motion planning and control framework presented in this paper was experimentally tested on the ballbot, a human-sized dynamically stable mobile robot that balances on a single ball. The results of successful experimental testing of two navigation tasks, namely, point-point and surveillance motions are presented. Additional experimental results that validate the framework’s capability to handle disturbances and rapidly replan in the presence of dynamic obstacles are also presented.
We address the problem of encoding and executing skills, i.e. motion tasks involving a combination of specifications regarding constraints and variability. We take an approach that is model-free in the sense that we do not assume an explicit and complete analytical specification of the task – which can be hard to obtain for many realistic robot systems. Instead, we learn an encoding of the skill from observations of an initial set of sample trajectories. This is achieved by encoding trajectories in a skill manifold which is learnt from data and generalizes in the sense that all trajectories on themanifold satisfy the constraints and allowable variability in the demonstrated samples. In new instances of the trajectory-generation problem, we restrict attention to geodesic trajectories on the learnt skill manifold, making computation more tractable. This procedure is also extended to accommodate dynamic obstacles and constraints, and to dynamically react against unexpected perturbations, enabling a form of model-free feedback control with respect to an incompletely modelled skill. We present experiments to validate this framework using various robotic systems – ranging from a three-link arm to a small humanoid robot – demonstrating significant computational improvements without loss of accuracy. Finally, we present a comparative evaluation of our framework against a state-of-the-art imitation-learning method.
This paper presents an algorithm for planning efficient trajectories in a bin-picking scenario. The presented algorithm is designed to provide paths, which are applicable for typical industrial manipulators, and does not require customized research interfaces to the robot controller. The method provides paths (almost) instantaneously, which is important for running efficiently in production. To achieve this, the method utilizes that all motions start and end within sub-volumes of the work envelope. A database of paths can thus be pre-computed, such that all paths are optimized with respect to a specified cost function, thereby ensuring close to optimal solutions. When queried, the method searches the database for a feasible path candidate and adapts it to the specific query. To achieve an efficient execution on the robot, blends are added to ensure a smooth transition between segments. Two algorithms for calculating feasible blends based on the clearance between robot and obstacles are therefore provided. Finally, the method is tested in a real bin-picking application where it solves queries efficiently and provides paths, which are significantly faster than those currently used for bin-picking in the industry.
Motion can be described in several alternative representations, including joint configuration or end-effector spaces, but also more complex topology-based representations that imply a change of Voronoi bias, metric or topology of the motion space. Certain types of robot interaction problems, e.g. wrapping around an object, can suitably be described by so-called writhe and interaction mesh representations. However, considering motion synthesis solely in a topology-based space is insufficient since it does not account for additional tasks and constraints in other representations. In this paper, we propose methods to combine and exploit different representations for synthesis and generalization of motion in dynamic environments. Our motion synthesis approach is formulated in the framework of optimal control as an approximate inference problem. This allows for consistent combination of multiple representations (e.g. across task, end-effector and joint space). Motion generalization to novel situations and kinematics is similarly performed by projecting motion from topology-based to joint configuration space. We demonstrate the benefit of our methods on problems where direct path finding in joint configuration space is extremely hard whereas local optimal control exploiting a representation with different topology can efficiently find optimal trajectories. In real-world demonstrations, we highlight the benefits of using topology-based representations for online motion generalization in dynamic environments.
Consider a thin, flexible wire of fixed length that is held at each end by a robotic gripper. Any curve traced by this wire when in static equilibrium is a local solution to a geometric optimal control problem, with boundary conditions that vary with the position and orientation of each gripper. We prove that the set of all local solutions to this problem over all possible boundary conditions is a smooth manifold of finite dimension that can be parameterized by a single chart. We show that this chart makes it easy to implement a sampling-based algorithm for quasi-static manipulation planning. We characterize the performance of such an algorithm with experiments in simulation.
We propose a method to plan optimal whole-body dynamic motion in multi-contact non-gaited transitions. Using a B-spline time parameterization for the active joints, we turn the motion-planning problem into a semi-infinite programming formulation that is solved by nonlinear optimization techniques. Our main contribution lies in producing constraint-satisfaction guaranteed motions for any time grid. Indeed, we use Taylor series expansion to approximate the dynamic and kinematic models over fixed successive time intervals, and transform the problem (constraints and cost functions) into time polynomials which coefficients are function of the optimization variables. The evaluation of the constraints turns then into computation of extrema (over each time interval) that are given to the solver. We also account for collisions and self-collisions constraints that have not a closed-form expression over the time. We address the problem of the balance within the optimization problem and demonstrate that generating whole-body multi-contact dynamic motion for complex tasks is possible and can be tractable, although still time consuming. We discuss thoroughly the planning of a sitting motion with the HRP-2 humanoid robot and assess our method with several other complex scenarios.
We consider the dynamic assignment and reassignment of a homogeneous robot ensemble to multiple spatially located tasks with deterministic or near-deterministic task execution times. Similar to Halasz et al. and Berman et al., we consider the development of agent-level, i.e. microscopic, stochastic control policies through the analysis of an appropriate macroscopic analytical model that describes the dynamics of the ensemble. Specifically, we present an approach to better approximate the effects of deterministic microscopic time delays at the macroscopic level based on Padé approximants. We present, analyze, and compare the frequency response of our approach to that presented by Berman et al. using different agent-based simulations.
We propose a combined spatial and non-spatial probabilistic modeling methodology motivated by an inspection task performed by a group of miniature robots. Our models explicitly consider spatiality and yield accurate predictions on system performance. An agent’s spatial distribution over time is modeled by the Fokker–Planck diffusion model and complements current non-spatial microscopic and macroscopic models that model the discrete state distribution of a distributed robotic system. We validate our models on a microscopic level based on sub-microscopic, embodied robot simulations as well as real robot experiments. Subsequently, using the validated microscopic models as our template, abstraction is raised to the level of macroscopic difference equations. We discuss the dependency of the modeling performance on the distance from the robot origin (drop-off location) and temporal convergence of the team distribution. Also, using an asymmetric setup, we show the necessity of spatial modeling methodologies for environments where the robotic platform underlies drift phenomena.