Run Sikuli in (J)Ruby - (J)IRB

I use irb for composing and debugging scripts. Following is what you might need to do inorder to run Sikuli in  IRB:

1. Install JRuby or via rvm
2. Copy sikuli-script.jar to  jruby lib
• cp /Applications/Sikuli-IDE.app/Contents/Resources/Java/sikuli-script.jar $JRUBY_HOME/lib
• For RVM:
• rvm list (to list currently installed rubies)
• rvm use jruby-1.x.x (to use jruby as the ruby implementation)
• rvm info (to print information about the ruby currently being used)
• cp /Applications/Sikuli-IDE.app/Contents/Resources/Java/sikuli-script.jar $MY_RUBY_HOME/lib ($MY_RUBY_HOME points to the current ruby home, in this case jruby)
3. run irb (the irb prompt should list jruby as the ruby implementation, something like jruby-1.x.x :001 >  )
4. jruby-1.x.x :001 > require 'java'                                                                                      
•  => true 
5. jruby-1.x.x :002 > java_import 'org.sikuli.script.Screen'                                                              
•  => Java::OrgSikuliScript::Screen 
6. jruby-1.x.x :003 > screen = Screen.new                                                                             
•  => #<Java::OrgSikuliScript::Screen:0x1dc2dad7>
7. jruby-1.6.0 :004 > image_path='/Users/mubbashir/Desktop'                                                             
•  => "/Users/mubbashir/Desktop"
8. screen.hover("#{image_path}/apple.png") 
• [info] Sikuli vision engine loaded.
•  => 1 

 You will see mouse pointer to be moved to apple icon.

With Auto Completion IRB is just a blessing, specially when you want to try some new library or need to run some snippets:

Similarly in script it would be something like:

	require 'java'
	java_import 'org.sikuli.script.Screen'
	screen = Screen.new
	image_path='/Users/mubbashir/Desktop'
	screen.click("#{image_path}/apple.png", 0)

view raw click_on_apple.rb hosted with ❤ by GitHub

to run the script ruby script_file_name.rb Just be sure that you are using JRuby ;)

Sikuli on Selenium- A demonstration of automation using selenium and Sikuli (such as flash uploader)

Selenium 2.0 is just launched, and the excitement remind me that there are tons of selenium related things I need to share with the world.

One of those things is File Upload. File uploads have always been a real pain to automate using selenium.

People have used AutoIt (so have I) to upload on windows only environment. On windows, beside AutoIt, native events  (of webdriver) have worked a great deal for me in the past. 

It becomes a real challenge  when file upload is done via some flash components (components which don't provides hooks from java script to play with files to upload).  

I had tired couple of things but the thing which worked really well was to have type="file"  and send keys but flash uploader and *nix remains challenging.  

Then I stumble across Sikuli. From there web site "Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots)".

Below is what I have done to use it to automat file upload via flash components on MacOS X. With few image changes we should be able to port it for Linux.

Source code is available at: https://github.com/mubbashir/Sikuli-on-Selenium

See it pretty smoothly uploads the file, but .. It works like a real user and real human user i.e. a real user can only interact with what is being displayed on the Screen. If the browser window dosen't have the focus Sikuli will not be able to find images.

Did I mention that test are written using TestNG, so this can slightly work as demonstration of TestNG on Selenium as well. 

Note: I haven't tried this in head-less mode.