A proposal of system for device capability assessment in bring your own device ( BYOD ) environment of computer assisted personal interviewing

Statistics Indonesia, known in Indonesia as Badan Pusat Statistik (BPS), has conducted a series of trials to utilize Computer-Assisted Personal Interview (CAPI) in a census or survey. However, CAPI costs a lot of money to procure and maintain the device. Bring Your Own Device (BYOD) principle offers cost opportunity to device management. In the practice of BYOD in BPS, a device that has good performance is required, because some devices aren’t capable of running CAPI properly. Yet, in BPS there is no standard value to qualify mobile devices to be used in census or survey. Therefore, it is necessary to review what kind of device is suitable for using CAPI. This study utilized CAPI used in a student’s field study as a case study reference. Initially, researchers develop benchmark applications as a tool for feasibility. Furthermore, developed application is tested on a certain base device to calculate the scores to be used as the standard values.


Introduction
Most of the censuses and surveys carried out by Statistics Indonesia, known as Badan Pusat Statistik (BPS), a government agency responsible for the official statistics in Indonesia, still utilize the Paper and Pencil Interviewing (PAPI) method in the data collection process.The PAPI method uses printed questionnaires and stationery as a tool for data collection.Census or survey generally begins with the preparation of the questionnaires.The stages of preparing the questionnaire require cost and time to print the questionnaire that has been designed.After that, the printed questionnaire will be distributed to each area covered in the census or survey.After the data is collected, there are still many stages that still need to be passed, including batching, cleaning, editing, and coding, until finally, the collected data can be input to the database.In the data entry process, more time and costs are needed for human resources and equipment.After that, validation needs to be done before the data sent to BPS headquarters to finally begin the compilation and tabulation process which also requires time and costs.Another shortcoming of PAPI is the handling of non-sampling errors that require additional effort from the employee to examine.
Consider the extensive stages that need to be resolved with PAPI and weighing the time and cost required, hence there is an urgency to apply another method as an alternative.One solution that can be applied is a Computer-Assisted Personal Interviewing (CAPI) method.The term CAPI refers to data collection by an in-person interviewer who use computers to administer the questionnaire to the respondent and captures the answers onto the computer [1].With the application of CAPI, the process of printing questionnaire will no longer be needed.Also, the process of batching, editing, and coding can be skipped so it can reduce the time and cost required when compared to PAPI.Another benefit of the CAPI is the possibility to apply to various validation rules directly to the device used in the data collection process.It is can be very helpful in an attempt to reduce non-sampling error.
Until now, BPS has conducted several preliminaries to use CAPI in various surveys.The first attempt to integrated CAPI in a survey was on 2011' Indonesian Tobacco Use Survey.An increase in the growth and development of mobile devices with vivid features and functionality has raised the bar to be used as an acceptable device within an organization [2], with no exception of BPS.These devices are now more powerful and sophisticated and have performance very near to that of desktops.In June 2013, BPS piloted a Survey of Knowledge, Attitudes, and Readiness to Face Disasters by using Androidbased tablets for field activities.The most recent trial in applying CAPI is the 2020 Population Census' pilot survey.However, the use of the CAPI method requires a substantial amount of costs for device procurement and management.BPS has estimated that the 2020 Population Census requires at least seven thousand units of the mobile device.While the number of devices owned by BPS is currently only around one thousand units.
To overcome the problems, BPS was planning to apply Bring Your Own Device (BYOD) in the upcoming Population Census.The BYOD concept refers to the application of employees' personal devices (such as laptops, tablets, smartphones, and other similar devices) to enable business services [3], which in this case the business is the data collection of census or survey using CAPI.BYOD has gained huge popularity and adoption.A survey conducted by Cisco in 2012 on 600 companies reveals that 95% of the surveyed companies are already permitting the use of personally owned devices in their work environments and assets [4].BYOD can provide profitable solutions for companies and employees [3].BYOD can provide several advantages, including asset management, cost advantages, employee satisfaction, and efficiency.
BYOD must be applied with careful planning so the advantages described previously can be achieved.In applying BYOD at BPS, a device that has a capable performance and resilience is required.The device that are going to be used should meet the certain requirement because not all devices are able to run CAPI software (from this point will be referred to briefly as CAPI) properly.There are several devices that experience problems when running CAPI, such as lag or force close.While currently at BPS there has not been such a standard value as a qualification, therefore, it is necessary to study what kind of equipment is suitable for using CAPI.
The aim of this study is to examine convenient measurement techniques to determine the standard value of a mobile device (smartphone or tablet computer) and determine the minimum specifications of a mobile device capable to run CAPI properly.Corresponded with the above objectives, this study also developed a benchmark application as a tool to measure the feasibility of a device in using CAPI.In computing, a benchmark is an act of running a program, a collection of programs or other similar processes to produce a relative performance value from an object, normally by running a number of standard tests and trials against it [5].The benchmark application was developed in the Android platform and will be called Android Benchmark.CAPI selected as the reference in this study was based on the Open Data Kit (ODK) Collect.ODK Collect is an open source Android app that replaces paper forms used in survey-based data gathering.Screenshots of the application can be seen in Fig. 1.

Materials and Methods
The assessment of employee device was intended to assure the quality of data produced by BPS regarding the plan of adopting BYOD in the data collection process.This section elaborates the methods and materials used in this research.

Workflow
The research was conducted by following the workflow as shown in Fig. 2. It was initially started by developing Android Benchmark as a tool to measure the performance of the certain component in a mobile device.After that, Android Benchmark was tested on a tablet Samsung Galaxy Tab A which served as a base device to provide a standard value for this research.This specific device was chosen as the base device due to its positive feedback obtained on a preliminary survey.The result from the experiment is then compared to auxiliary information to attain the accuracy of the measurement.The auxiliary information consists of two variables, the first is how the user percepts performance of his/her device and the second is empirical performance of the devices captured by questions asked to the owners about their experience (e.g.how long it takes to load a questionnaire or how often lag occurred when navigating through questionnaire) when using CAPI with their devices.Each variable was binary consists of "good" and "bad".By comparing assessment result to the auxiliary variable, the validity of the assessment can be estimated.

Android Benchmark
Android Benchmark consists of four benchmarks as described below: 1. CPU Benchmark This benchmark evaluates the performance of the Central Processing Unit (CPU).It can be divided into three small benchmarks: a. Integer Arithmetic, it stresses the Arithmetic Logic Unit (ALU) by doing lots of integer computations; b.Floating Point Arithmetic, it stresses the Floating Point Unit (FPU) by doing lots of floating point computations; c.Digits of PI, it tests caching and multithreading by computing digits of PI, using the Bailey-Borwein-Plouffe formula in a parallel manner.

Hashing Benchmark
This benchmark computes hashes using the BCrypt Algorithm.

Files Benchmark
This benchmark evaluates the read and write performance (Storage I/O) of the device using a fixed 4 kilobytes buffer.

Network Benchmark
This benchmark measures download speed by downloading part of a large file.

Babylonian Method of Computing the Square Root
When computing a square root, computers still, in effect, use an iterative algorithm developed by the ancient Babylonians millennia ago.This algorithm proved to be most efficient compared to others and can be explained as the computationally simplest techniques [6].For instance, to compute the square root of  = √, it starts with an arbitrary positive number  0 , and then apply the following iterative process:

Bailey-Borwein-Plouffe Formula
Bailey-Borwein-Plouffe (BBP) is a formula to compute a certain digit of mathematical constant π.The formula is: The formula can directly calculate the value of any given digit of π without calculating the preceding digits.BBP formula works in an algorithm that can be easily implemented, require virtually no memory, and feature run times that scale nearly linearly with the order of the digit desired [7].

Profiling Devices
The number of devices used in this research' experiment was thirty-three Android devices that have different hardware configurations to gather the performance data using Android Benchmark.Table 1 specifies some of the characteristics of the evaluated Android devices.

Results and Discussion
Android Benchmark can measure the basic performance of each component and the overall performance of a device.The performance will be displayed in a set of scores.The higher the score achieved by a device means the better the performance of that device.The range of score may vary depends on the type of the benchmark.

Standard Values
The result of the test conducted on the base device was presented in Table 2.There are three units of Samsung Galaxy Tab A used in the test.The overall score was derived from the geometric mean of the other four benchmark score.From the result of the base device, the standard values were obtained by averaging the score.

Test Result on Sample Devices
The results presented are device performance measurement of sample devices and are the average values from the test repeated three times.These include CPU benchmark, hashing benchmark, file benchmark, and overall benchmark.
The CPU Benchmark is composed of three smaller benchmarks, namely integer arithmetic, floating point arithmetic, and digits of PI.These benchmarks were executed separately and in sequence.Generally, to calculate the score for both integer arithmetic and floating-point arithmetic, the algorithm used is shown in Fig. 3. To calculate the integer arithmetic score, the device was assigned to do a lot of fixed-point operations using integer data types.Whilst to calculate floating point arithmetic score, the device was assigned to approximate certain number' square roots using the Babylonian method in (1).To calculate the score of digits of PI followed different algorithm presented in Fig. 4. using BBP formula in (2).After the scores of three smaller benchmarks were obtained, these scores were combined to CPU Benchmark score using geometric means.The result is shown in Fig. 5.
The Hashing Benchmark uses the built-in function of Bcrypt hash calculator [8].The function is an implementation of the OpenBSD Blowfish password hashing algorithm.The algorithm to calculate the Hashing Benchmark score follows steps as shown in Fig. 6.The result of Hashing Benchmark generated by sampled devices is presented in Fig. 7.
Files Benchmark calculated devices' storage speed in performing file write and read.The score was generated following algorithm is shown in Fig. 8. Files Benchmark score is derived from averaging the write speed and the read speed of a device.The result from sampled devices is presented in Fig. 9.

Validity Test
To get the validity of the performance measurement, the result of overall performance modified to the binary category of "good" and "bad".A device will be classified as good if the device got an overall score above or equal the standard value, otherwise, if a device got an overall score below standard value then it classified as bad.The result of the validity test is shown in Table 3.
It can be determined from the results that the standard values for a device to run CAPI were 2099 for overall score measured by Android Benchmark.Thereupon, the standard values for individual benchmark were 6870, 3457, 1978, and 444 respectively for CPU, Hashing, Files, and Network.By comparing with auxiliary information, this research obtained the accuracy for measurement of the developed benchmark application.The validity test shows a result of 76,7% when compared to user perception, while the validity test shows accuracy of 80% when compared to the user experience.For addition, it is necessary to emphasize that this standard value can only be applied to assessing device which going to run the reference CAPI.This research served as a proposal on adopting BYOD in BPS.To determine the standard for another type of CAPI there is a necessity to conduct another comprehensive study.

Conclusion
This study utilized CAPI used in a student's field study as a case study reference.The benchmark applications as a tool for feasibility was developed.Furthermore, the developed application was successfully tested on a certain base device to calculate the scores to be used as the standard values.For future research, development of the standard value needs to be done so the assessment of device can be executed regardless of the CAPI that will be used in the data collection process.Literature shows that data leakage, distributed denial of service (DDoS), and malware are the most challenging security threats to BYOD [9], so there is an urge for future research to put security issues in consideration.

Fig. 2 .
Fig. 2. WorkflowThen, we conducted an experiment of the Android Benchmark on several different devices.The experiment was done in the Campus of Politeknik Statistika STIS and the devices involved were sampled from the student that had previously use CAPI on their device.The total of device experimented on were thirty Android smartphones which made by several manufacturers and consists of various hardware and software specifications.The result from the experiment is then compared to auxiliary information to attain the accuracy of the measurement.The auxiliary information consists of two variables, the first is how the user percepts performance of his/her device and the second is empirical performance of the devices captured by questions asked to the owners about their experience (e.g.how long it takes to load a questionnaire or how often lag occurred when navigating through questionnaire) when using CAPI with their devices.Each variable was binary consists of "good" and "bad".By comparing assessment result to the auxiliary variable, the validity of the assessment can be estimated.

Fig. 4 .
Fig. 4. Algorithm to calculate score for digits of PI

Table 1 .
Characteristics of evaluated devices

Table 2 .
Test result on base device

Table 3 .
Result of validity test