Vehicles with advanced driving assist systems that automatically steer, accelerate and brake are popular, but associated with increased driver distraction. This distraction coupled with unreliable autonomous system performance leads to vehicles that may be at higher risk for striking pedestrians. To this end, this study tested three consumer vehicles in two different model classes in a pedestrian crossing scenario. In 120 trials, one model never detected the pedestrian, nor alerted the driver. In 123 trials, the other model vehicles almost always detected the pedestrian, but in 35% of trials, alerted the driver too late. These cars were not consistent internally or with one another in pedestrian detection and response, and only sparingly sounded any warnings. These intelligent vehicles also detected the pedestrian earlier if there were no established lane lines, suggesting that in well-marked areas, typically the case in for established crossings, pedestrians may be at increased risk of a possible conflict. This research demonstrates that artificial intelligence can lead to unreliable vehicle behaviors and warnings in pedestrian detection, potentially catching drivers off guard. These results further indicate industry needs to do more testing of intelligent systems, regulators should reevaluate the self-certification approval process, and that more fundamental work is needed in academia around the performance and quality of technologies with embedded neural networks.