Warning: Declaration of P_Photocrati_NextGen::define() should be compatible with C_Base_Product::define($id = 'pope-produ...', $name = 'Pope Produ...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/product.photocrati_nextgen.php on line 0

Warning: Declaration of M_Fs::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/fs/module.fs.php on line 0

Warning: Declaration of M_Router::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/router/module.router.php on line 0

Warning: Declaration of M_I18N::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/i18n/module.i18n.php on line 0

Warning: Declaration of M_Validation::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/validation/module.validation.php on line 0

Warning: Declaration of M_WordPress_Routing::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/wordpress_routing/module.wordpress_routing.php on line 0

Warning: Declaration of M_Security::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/security/module.security.php on line 0

Warning: Declaration of M_Lzw::define($context = false) should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/lzw/module.lzw.php on line 0

Warning: Declaration of M_NextGen_Settings::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_settings/module.nextgen_settings.php on line 0

Warning: Declaration of M_MVC::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/mvc/module.mvc.php on line 0

Warning: Declaration of M_Ajax::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/ajax/module.ajax.php on line 0

Warning: Declaration of M_Dynamic_Stylesheet::define($context = false) should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/dynamic_stylesheet/module.dynamic_stylesheet.php on line 0

Warning: Declaration of M_Frame_Communication::define($context = false) should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/frame_communication/module.frame_communication.php on line 0

Warning: Declaration of M_DataMapper::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/datamapper/module.datamapper.php on line 0

Warning: Declaration of M_NggLegacy::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/ngglegacy/module.ngglegacy.php on line 0

Warning: Declaration of M_NextGen_Data::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_data/module.nextgen_data.php on line 0

Warning: Declaration of M_Dynamic_Thumbnails::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/dynamic_thumbnails/module.dynamic_thumbnails.php on line 0

Warning: Declaration of M_NextGen_Admin::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_admin/module.nextgen_admin.php on line 0

Warning: Declaration of M_NextGen_AddGallery_Page::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_addgallery_page/module.nextgen_addgallery_page.php on line 0

Warning: Declaration of M_NextGen_Pagination::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_pagination/module.nextgen_pagination.php on line 0

Warning: Declaration of M_Gallery_Display::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_gallery_display/module.nextgen_gallery_display.php on line 0

Warning: Declaration of M_Attach_To_Post::define($context = false) should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/attach_to_post/module.attach_to_post.php on line 0

Warning: Declaration of M_NextGen_Other_Options::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_other_options/module.nextgen_other_options.php on line 0

Warning: Declaration of M_NextGen_Pro_Upgrade::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_pro_upgrade/module.nextgen_pro_upgrade.php on line 0

Warning: Declaration of M_MediaRss::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/mediarss/module.mediarss.php on line 0

Warning: Declaration of M_Cache::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/cache/module.cache.php on line 0

Warning: Declaration of M_Lightbox::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/lightbox/module.lightbox.php on line 0

Warning: Declaration of M_NextGen_Basic_Album::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_album/module.nextgen_basic_album.php on line 0

Warning: Declaration of C_NextGen_Basic_Album_Installer::install() should be compatible with C_Gallery_Display_Installer::install($reset = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_album/class.nextgen_basic_album_installer.php on line 0

Warning: Declaration of M_NextGen_Basic_Templates::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_templates/module.nextgen_basic_templates.php on line 0

Warning: Declaration of M_NextGen_Basic_Gallery::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_gallery/module.nextgen_basic_gallery.php on line 0

Warning: Declaration of C_NextGen_Basic_Gallery_Installer::install() should be compatible with C_Gallery_Display_Installer::install($reset = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_gallery/class.nextgen_basic_gallery_installer.php on line 0

Warning: Declaration of M_NextGen_Basic_ImageBrowser::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_imagebrowser/module.nextgen_basic_imagebrowser.php on line 0

Warning: Declaration of C_NextGen_Basic_ImageBrowser_Installer::install() should be compatible with C_Gallery_Display_Installer::install($reset = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_imagebrowser/class.nextgen_basic_imagebrowser_installer.php on line 0

Warning: Declaration of M_NextGen_Basic_Singlepic::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_singlepic/module.nextgen_basic_singlepic.php on line 0

Warning: Declaration of C_NextGen_Basic_SinglePic_Installer::install() should be compatible with C_Gallery_Display_Installer::install($reset = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_singlepic/class.nextgen_basic_singlepic_installer.php on line 0

Warning: Declaration of M_NextGen_Basic_Tagcloud::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_tagcloud/module.nextgen_basic_tagcloud.php on line 0

Warning: Declaration of C_NextGen_Basic_Tagcloud_Installer::install() should be compatible with C_Gallery_Display_Installer::install($reset = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_basic_tagcloud/class.nextgen_basic_tagcloud_installer.php on line 0

Warning: Declaration of M_Widget::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/widget/module.widget.php on line 0

Warning: Declaration of M_Third_Party_Compat::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/third_party_compat/module.third_party_compat.php on line 0

Warning: Declaration of M_NextGen_XmlRpc::define() should be compatible with C_Base_Module::define($id = 'pope-modul...', $name = 'Pope Modul...', $description = '', $version = '', $uri = '', $author = '', $author_uri = '', $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_xmlrpc/module.nextgen_xmlrpc.php on line 0

Warning: session_start(): Cannot send session cookie - headers already sent by (output started at /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/product.photocrati_nextgen.php:0) in /home/sidesul6/public_html/wp-content/plugins/one-time-password/wp-otp-class.php on line 78

Warning: session_start(): Cannot send session cache limiter - headers already sent (output started at /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/product.photocrati_nextgen.php:0) in /home/sidesul6/public_html/wp-content/plugins/one-time-password/wp-otp-class.php on line 78

Warning: Declaration of Walker_PostExpirator_Category_Checklist::start_lvl(&$output, $depth, $args) should be compatible with Walker::start_lvl(&$output, $depth = 0, $args = Array) in /home/sidesul6/public_html/wp-content/plugins/post-expirator/post-expirator.php on line 0

Warning: Declaration of Walker_PostExpirator_Category_Checklist::end_lvl(&$output, $depth, $args) should be compatible with Walker::end_lvl(&$output, $depth = 0, $args = Array) in /home/sidesul6/public_html/wp-content/plugins/post-expirator/post-expirator.php on line 0

Warning: Declaration of Walker_PostExpirator_Category_Checklist::start_el(&$output, $category, $depth, $args) should be compatible with Walker::start_el(&$output, $object, $depth = 0, $args = Array, $current_object_id = 0) in /home/sidesul6/public_html/wp-content/plugins/post-expirator/post-expirator.php on line 0

Warning: Declaration of Walker_PostExpirator_Category_Checklist::end_el(&$output, $category, $depth, $args) should be compatible with Walker::end_el(&$output, $object, $depth = 0, $args = Array) in /home/sidesul6/public_html/wp-content/plugins/post-expirator/post-expirator.php on line 0
Remotely Control iPhone via SSH and VNC | SideApps
Warning: Declaration of C_DataMapper_Driver_Base::define($object_name, $context = false) should be compatible with C_Component::define($context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/datamapper/class.datamapper_driver_base.php on line 0

Warning: Declaration of C_DataMapper_Model::__set($property_name, $value) should be compatible with & ExtensibleObject::__set($property, $value) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/datamapper/class.datamapper_model.php on line 0

Warning: Declaration of C_Lightbox_Library_Mapper::define($context = false) should be compatible with C_CustomPost_DataMapper_Driver::define($object_name, $context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/lightbox/class.lightbox_library_mapper.php on line 0

Warning: Declaration of C_Gallery_Storage::define($object_name, $context = false) should be compatible with C_Component::define($context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_data/class.gallery_storage.php on line 42

Warning: Declaration of C_GalleryStorage_Driver_Base::define($context) should be compatible with C_Component::define($context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/nextgen_data/class.gallerystorage_driver_base.php on line 0

Warning: Declaration of C_MVC_View::define($template, $params = Array, $engine = 'php', $context = false) should be compatible with C_Component::define($context = false) in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/mvc/class.mvc_view.php on line 0

Warning: Declaration of C_MVC_View::initialize($template, $params = Array, $engine = 'php', $context = false) should be compatible with C_Component::initialize() in /home/sidesul6/public_html/wp-content/plugins/nextgen-gallery/products/photocrati_nextgen/modules/mvc/class.mvc_view.php on line 0

Remotely Control iPhone via SSH and VNC

This is an issue I spent far too much time time on. I stumbled upon a great new game, Threes, and had become consumed with the internal workings – how the game chooses what card to produce, where to place it, etc. In order to get some real data to work with I needed to complete a multitude of games, all while keeping track of which cards were produced and where. This is the type of menial task I have no patience for and I quickly began to seek a method to do this automatically.

I had a jailbroken phone – something you will need to do have if you are interested in any of the methods I am about to describe. This method is 99.9% effective and, if something goes wrong, you can completely fix it by restoring through iTunes. Unlocking your phone is dangerous – it has the possibility of leading to a “bricked” iPhone. Jailbreaking is not unlocking. If you’re interested in jailbreaking check out evasi0n – the latest version of their tool is working on iOS 7.0.6.

With that out of the way, here’s what you’ll need and why:

  • OpenSSH: SSH is ubiquitous method of remotely accessing another system through the command line. This is how we’ll interact with the iPhone and send it the commands we want.
  • SimulateTouch: This package allows you to (you guessed it) send touch events to your iPhone using command line tools.
  • Optional Veency: This is a VNC server that runs on the iPhone. VNC is another tool that allows you to remotely access devices. However, unlike SSH, it allows you to see the screen of the remote device and send things like touch and keyboard events. But I thought we were using SimulateTouch to send touch events? Yes, that’s right. Though Veency can be used this way as well I’ve found it to be far less reliable than the SimulateTouch library. Sometimes the commands I send are just lost in the ether.
  • Optional Activator: This allows you to set up a handful (read: a ton) of custom gestures and actions that can do anything from launch applications to disable WiFi to restart your iPhone. It also, conveniently, has a command line interface which can be used to activate any of its abilities via SSH – no gestures needed.

That’s it for the software! Everything I just listed can be downloaded, for free, through Cydia. Cydia is installed automatically when you jailbreak and is the de facto way of installing 3rd party software.

So, now that we have all these tools – how do we use them?! I’m a big python buff so all of my examples are going to be in python. You can use any language you like, but you’ll have to interpret the mysterious code I provide you.

Before I start with examples, here’s the biggest pitfall of our setup:

We don’t have a way to view the current screen in applications that run at 30 FPS or more. Veency simply can’t keep up and provides a distorted display. I’ll explain how I’ve been getting around this in my examples below.

Connecting via SSH:

I’m using Spur as my SSH interface. You can use whatever you want. We use the username ‘root’ because we want full access to the iPhone filesystem. The default SSH_PASSWORD is ‘alpine’ (and has been forever) – I recommend you change this as it is somewhat of a security issue.

Connecting via VNC:

Here I’m using the vncDoTool. Again, use what you want. You can set up the VNC_PASSWORD in the Veency settings.

Sending Touch Events:

The SimulateTouch library makes this dead simple. My example above shows how to make a swipe in any direction. You provide the X,Y coordinates of the start position and the end position. The last two commands are the duration of the swipe and the orientation of the device (1 = portrait).

Taking a ScreenShot:

My first example instructs Activator to take a screenshot. My second example uses VNC to hold down the power button, press the home button, and release the power button – all in quick succession. As you probably know, this is the built in way to take screenshots in iOS. These screenshots will be stored in your camera roll.

Accessing the ScreenShot:

This is a little more complicated. What we’re doing is listing the files in our screenshot directory by their creation time (yours may not be 108APPLE but it will be in the same DCIM directory). We grab the first of these files and assume (correctly) that it’s the screenshot we just took. We then open this file with SSH – this effectively stores the image in memory. Then we use python to open a file for writing – this is where we’ll store the screenshot. We use shutil to copy the screenshot from memory to the file location we opened. Finally, we close out both of our file streams.

In the second to last command I delete the screenshot file. Unfortunately, Apple has made their screenshot system ridiculously complicated. If you navigate to your Camera Roll you’ll find your image is still there. Click on it and a blank screen will be loaded. We’ve deleted the image file it’s trying to load but haven’t gotten rid of the image reference or the associated thumbnail that was created. I haven’t found a way around this.

The final command stores our image into the program memory so I can use it elsewhere without having to read the file from disk each time.

Well, this concludes this fairly verbose tutorial. If you have any questions I’ll do my best to answer them in the comments. I’ll leave you with close to 1000 lines of Python code for you to peruse. This is the code I came up with for automating the gameplay of the addictive and beautiful new game Threes.

 

Leave a Reply

Your email address will not be published. Required fields are marked *


Warning: Parameter 1 to W3_Plugin_TotalCache::ob_callback() expected to be a reference, value given in /home/sidesul6/public_html/wp-includes/functions.php on line 3297