To test IOS devices

Hello everyone:

We are setting up a private instance to test performance on ipad and iphone. We can see a few instructions for IOS devices on https://sites.google.com/a/webpagetest.org/docs/private-instances/node-js-agent/setup , but I meet a problem when I try the command:

wptdriver.sh
-m debug
–browser ios:0088a434deadbeef
–serverUrl example.com
–location Ipad
–processvideo yes

I have to set a browser, I opened wptdriver.sh and tried to read. I find it seems there’s no according browser for ios devices, but only for addroid and chrome…

Does it mean we have to use mobitest agent to support IOS devices?

I’ve made this configuration in locations.ini on my host:

[locations]
11=Office

[Office]
1=Ipad

[Ipad]
browser=“Safari”
label=“Safari”
type=“node.js,mobile”
connectivity=“Wifi”

Thanks for any help

Mobitest is an app that polls the WPT server for work, and then loads the page in a WebView.

The NodeJS and iOS device are a better equivalent, and only launch Safari

Also, did you use that wptdriver.sh command literally? You need to replace the device ID with the ID of your actual device and the server URL with your actual server.

Thanks for your remind, I will replace the device ID and Url.
What makes me most confused is the browser argument, we can see your code of wptdriver.bat:

echo -b, --browser VALUE Browser type, which must be one of:
echo chrome # Local Chrome browser
echo android:DID # Android device id
echo Defaults to ‘chrome’.

We’d like to make tests on Ipad safari. Is there an option for Ipad safari?

Thanks,

Yes, the ios: line you have above will launch Safari on an iOS device (iPad included). The iOS support is just becoming stable though it’s still under VERY active development so don’t be surprised if there are a few rough edges (and a lot more limitations than the android testing).

Hmm… But from wptdriver.bat, it seems only launch chrome, could you please show me how to lauch Safari?

Thanks,

–browser chrome - launches Chrome on the local desktop (support is really sketchy and not well tested)

–browser android:DID - launches Chrome on the android device specified by DID (should be pretty stable and is the best supported from the node agent)

–browser ios:DID - launches Safari on the iOS device specified by DID (Should work though I just started using it this week on the public instance and there are still a lot of rough edges - under heavy active development as issues are discovered)

Hmm… I still get “Unknown browser ios”
The command I’m trying is:
pathTo\wptdriver.bat -m debug --browser iso:DMPKJ086FAKE --serverUrl http://myurl --location Ipad --processvideo yes -apiKey=my_api_key

Need I update my wptdriver somewhere? I’m using the release 2.18.

Thanks,

If you want to test iOS you’ll probably want to grab from trunk on github as I’ve been doing a lot of work on it over the last 2 weeks.

That said, that device ID doesn’t look right - the device ID’s are like 40-character strings. Once you grab from trunk you can see a list of the attached device ID’s by running “lib/ios/video/xrecord -q -l”

Exactly, my DID is not correct. Thank you.

We find the code in wptdriver.bat are less developed than wptdriver.sh, and this is why my -ios argument saying invalide browser. I have made some changes to have the ios argument:

[code]for %%I in (%AGENT%\lib\ios\openURL\openURL*.ipa) do set URL_APP=%%I
if defined URL_APP set “URL_APP_ARGS= --iosUrlOpenerApp ^”%URL_APP%^“”

if “%BROWSER%”==“ios” (
set KNOWN_BROWSER=1
set “BROWSER_ARGS= --browser browser_ios.BrowserIos --deviceSerial %DEVICE_SERIAL% --captureDir ^”%AGENT%\lib\capture^" --iosIDeviceDir ^“%AGENT%\lib\ios\idevice\Darwin x86_64^” --iosDevImageDir ^“%AGENT%\lib\ios\DeviceSupport”^ --iosSshProxyDir ^“%AGENT%\lib\ios\usbmux_python_client”^ --iosVideoDir ^“%AGENT%\lib\ios\video^” %URL_APP_ARGS%"
)[/code]

We run the wptdriver.bat command but get:

W Sep_15_11:38:03.375 C:\webpagetest\mobile\src\wpt_client.js:512 Client.<anonym ous> : Agent is not ready: ssh -F /dev/null -i undefined/.ssh/id_dsa_ios -o 'Pro xyCommand="^C:\webpagetest\mobile\\lib\ios\usbmux_python_client^/sshproxy.py" -u %h' -o 'User=root' 395d0d4e30d832a2b94c0fb6fac9cd6999064698 'echo show State:/N etwork/Interface/en0/IPv4|scutil' failed, code -4058

From the error message, I can see a ssh command. I just suspect if the ios devices tests are only supported on Linux OS at this moment? I have to set up a linux agent to test ios devices?

For the ssh keys, I generated and put the private key on my ipad ~/.ssh/id_dsa_ios, and put the public key on my windows c:/Users/webpagetest/.ssh/authorized_keys

Thanks,

Oh yeah, you’re not going to have a good time if you try to use Windows. Not only the ssh but also the ios_webkit_debug_proxy doesn’t work on Windows. Linux and Mac OSX are the only supported platforms and only Mac OSX can be used if you want video capture.

Thanks for your advise.
I set up a Mac host today following the documentation. It looks everything work fine, except a timeout debug information.
My web server is able to list my iPad in the location dropdownlist, and saying my test is on the front of the queue, but can’t proceed the test.
I can’t see the browser being opened on the iPad.

Here is the output from my Terminal on Mac:

[quote]node --max-old-space-size=4096 --expose-gc src/agent_main --serverUrl http://myurl.net --location Ipad --browser browser_ios.BrowserIos --deviceSerial 395d0d4e30d832a2b94c0fb6fac9cd6999064698 --captureDir /Users/c_liu/webpagetest/agent/js/lib/capture --iosIDeviceDir /Users/c_liu/webpagetest/agent/js/lib/ios/idevice/Darwin x86_64 --iosDevImageDir /Users/c_liu/webpagetest/agent/js/lib/ios/DeviceSupport --iosSshProxyDir /Users/c_liu/webpagetest/agent/js/lib/ios/usbmux_python_client --iosVideoDir /Users/c_liu/webpagetest/agent/js/lib/ios/video --iosUrlOpenerApp /Users/c_liu/webpagetest/agent/js/lib/ios/openURL/openURL*.ipa --processvideo yes
D Sep_16_15:10:02.432 browser_base.js:141 createBrowser : Creating browser browser_ios.BrowserIos
I Sep_16_15:10:02.453 browser_ios.js:58 new BrowserIos : BrowserIos({“flags”:{“serverUrl”:“http://myurl.net”,“location”:“Ipad”,“browser”:“browser_ios.BrowserIos”,“deviceSerial”:“395d0d4e30d832a2b94c0fb6fac9cd6999064698”,“captureDir”:“/Users/c_liu/webpagetest/agent/js/lib/capture”,“iosIDeviceDir”:“/Users/c_liu/webpagetest/agent/js/lib/ios/idevice/Darwin x86_64”,“iosDevImageDir”:“/Users/c_liu/webpagetest/agent/js/lib/ios/DeviceSupport”,“iosSshProxyDir”:“/Users/c_liu/webpagetest/agent/js/lib/ios/usbmux_python_client”,“iosVideoDir”:“/Users/c_liu/webpagetest/agent/js/lib/ios/video”,“iosUrlOpenerApp”:“/Users/c_liu/webpagetest/agent/js/lib/ios/openURL/openURL*.ipa”,“processvideo”:“yes”},“task”:{}})
D Sep_16_15:10:02.481 process_utils.js:487 unknown : Calling Make dirs
D Sep_16_15:10:02.483 process_utils.js:481 cb : Callback for Make dirs
D Sep_16_15:10:02.495 process_utils.js:487 unknown : Calling Work read
D Sep_16_15:10:02.497 process_utils.js:481 cb : Callback for Work read
D Sep_16_15:10:02.550 process_utils.js:263 unknown : Exec with timeout(10000): ssh -F /dev/null -i /Users/c_liu/.ssh/id_dsa_ios -o ‘ProxyCommand=“/Users/c_liu/webpagetest/agent/js/lib/ios/usbmux_python_client/sshproxy.py” -u %h’ -o ‘User=root’ 395d0d4e30d832a2b94c0fb6fac9cd6999064698 ‘echo show State:/Network/Interface/en0/IPv4|scutil’
D Sep_16_15:10:02.893 process_utils.js:327 ChildProcess. : stdout[177] {
Addresses : {
0 : 10.100.40.54
}
BroadcastAddresses : {
0 : 10.100.43.255
}
SubnetMasks : {
0 : 255.255.252.0
}
}
, stderr[161] Connecting to device <MuxDevice: ID 1 ProdID 0x12ab Serial ‘395d0d4e30d832a2b94c0fb6fac9cd6999064698’ Location 0x1d110000>
Connection established, relaying data
[color=#FF0000]D Sep_16_15:10:02.900 process_utils.js:263 unknown : Exec with timeout(10000): ssh -F /dev/null -i /Users/c_liu/.ssh/id_dsa_ios -o ‘ProxyCommand=“/Users/c_liu/webpagetest/agent/js/lib/ios/usbmux_python_client/sshproxy.py” -u %h’ -o ‘User=root’ 395d0d4e30d832a2b94c0fb6fac9cd6999064698 killall MobileSafari[/color]
I Sep_16_15:10:03.236 wpt_client.js:490 Client. : Get work: http://myurl.net/work/getwork.php?location=Ipad&pc=395d0d4e30d832a2b94c0fb6fac9cd6999064698&f=json
[/quote]

Is this timeout the problem?

Thanks,

Is your server actually at “myurl.net”?

http://myurl.net/work/getwork.php?location=Ipad&pc=395d0d4e30d832a2b94c0fb6fac9cd6999064698&f=json

If so, what does the locations.ini look like? Looks like the agent is working fine in talking to the device so now it’s just making sure the config matches what is defined on the server.

Nope, it’s not my real url :stuck_out_tongue:
But we can see it’s in the good format, and my other devices are working fine :smiley:

Here is the Ipad part of my locations.ini

[hr]
I just find my ipad safari browser can only stay open several seconds.
It’s able to be closed by the script running on the host.
[hr]
The bad part is the Ipad becomes to inactivated suddenly…
Office : Office (Chrome, Firefox, Canary, IE 11, Nexus5 - Chrome, Galaxy S5 - Chrome, Ipad)
Office_wpt : Office - 2 agents connected
Mobile_Nexus5 : Nexus 5 - 1 agents connected
Mobile_GalaxyS5 : Galaxy S5 - 1 agents connected
Ipad : Ipad - 274 minutes since last agent connected

Do you see anything in the debug output on the terminal where the iOS agent is running to see why it may have gone offline? An unhandled exception, node crashing, etc? I run my agent in a while loop from bash to auto-restart it if it exited but there are also cases where sometimes it just gets wedged (been fixing those over the last week or two).

url: http://myrul.net/work/getwork.php?location=Ipad&pc=395d0d4e30d832a2b94c0fb6fac9cd6999064698&f=json

I find the problem: I have key settings on my web server.
When the server get the request, the key parameter from url is null, but the key in my settings.ini is a 40 characters string.
So, my question will be how can I add key parameter into the url? I try to add --key, -key and --apiKey, but they won’t work. ( apiKey works for my android devices)
[hr]

Nothing more than a timeout message:

Thanks for your advise. If something wrong happens, your loop is a good try.
[hr]
–apiKey does work for sending request! I guess I made a typo on last try

[quote]Office : Office (Chrome, Firefox, Canary, IE 11, Nexus5 - Chrome, Galaxy S5 - Chrome, Ipad, iPhone)
Office_wpt : Office - 2 agents connected
Mobile_Nexus5 : Nexus 5 - 1 agents connected
Mobile_GalaxyS5 : Galaxy S5 - 1 agents connected
Ipad : Ipad - 1 agents connected[/quote]
[hr]
The test can proceed now, however, the result is always not successful…

Let me try to find something from the output…

I believe it’s not a good experience to read the whole output :smiley:

I just take some suspicious parts from it:

[quote]…
D Sep_17_15:34:38.965 process_utils.js:263 unknown : Exec with timeout(10000): ./ipfw_config clear --device 395d0d4e30d832a2b94c0fb6ddd9cd6999064698
D Sep_17_15:34:39.010 process_utils.js:327 ChildProcess. : stderr[116] +++ /sbin/ipfw list
ipfw: socket: Operation not permitted
+++ /sbin/ipfw list
ipfw: socket: Operation not permitted
D Sep_17_15:34:39.665 browser_base.js:141 createBrowser : Creating browser browser_ios.BrowserIos

Connection established, relaying data
D Sep_17_15:34:40.083 process_utils.js:263 unknown : Exec with timeout(10000): ‘/Users/c_liu/webpagetest/agent/js/lib/ios/idevice/Darwin x86_64/ideviceinfo’ -k ProductVersion -u 395d0d4e30d832a2b94c0fb6ddd9cd6999064698
D Sep_17_15:34:40.410 process_utils.js:327 ChildProcess. : stdout[4] 8.3
E Sep_17_15:34:40.426 wd_server.js:204 WebDriverServer. : Exception from “Run test”: Error: Missing Xcode image: /Users/c_liu/webpagetest/agent/js/lib/ios/DeviceSupport/8.3/DeveloperDiskImage.dmg{,.signature}
D Sep_17_15:34:40.427 wd_server.js:205 WebDriverServer. : Error: Missing Xcode image: /Users/c_liu/webpagetest/agent/js/lib/ios/DeviceSupport/8.3/DeveloperDiskImage.dmg{,.signature}
at reject (/Users/c_liu/webpagetest/agent/js/src/browser_ios.js:138:46)
at BrowserIos. (/Users/c_liu/webpagetest/agent/js/src/browser_ios.js:154:13)
at FSReqWrap.cb [as oncomplete] (fs.js:226:19)

E Sep_17_15:34:40.589 wd_server.js:204 WebDriverServer. : Exception from “Capture Screen Shot”: Error: ‘/Users/c_liu/webpagetest/agent/js/lib/ios/idevice/Darwin x86_64/idevicescreenshot’ -u 395d0d4e30d832a2b94c0fb6ddd9cd6999064698 failed, code 255, stdout[153] Could not start screenshotr service! Remember that you have to mount the Developer disk image on your device if you want to use the screenshotr service.
D Sep_17_15:34:40.590 wd_server.js:205 WebDriverServer. : Error: ‘/Users/c_liu/webpagetest/agent/js/lib/ios/idevice/Darwin x86_64/idevicescreenshot’ -u 395d0d4e30d832a2b94c0fb6ddd9cd6999064698 failed, code 255, stdout[153] Could not start screenshotr service! Remember that you have to mount the Developer disk image on your device if you want to use the screenshotr service.

at newError (/Users/c_liu/webpagetest/agent/js/src/process_utils.js:279:17)
at ChildProcess.<anonymous> (/Users/c_liu/webpagetest/agent/js/src/process_utils.js:325:23)
at ChildProcess.emit (events.js:110:17)
at maybeClose (child_process.js:1015:16)
at Socket.<anonymous> (child_process.js:1183:11)
at Socket.emit (events.js:107:17)
at Pipe.close (net.js:485:12)


[/quote]

I have copied /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform to /DeviceSupport to webpagetest/agent/js/lib/ios/DeviceSupport
and checked the subdirectories are like :

[quote]4.2 5.1 7.0 8.1
4.3 6.0 7.1 8.2 (12D508)
5.0 6.1 8.0[/quote]

Maybe I need to update my Xcode.app somewhere…
But we can see it’s also complaining an ipfw problem… Could you help me on it?

Thanks,

I’m so happy to anounce:

after updated Xcode.app, and copied all the DeviceSupport files. My iPad is able to work!

Thanks Patrick